Systems and methods for determining relative placement of content items on a rendered page

ABSTRACT

Systems and methods for determining the value of bids placed by content providers for placement positions on a page, e.g., a web page, rendered according to a given context, for instance, the search results listing for a particular query initiated on a search engine web site, are provided. Additionally, systems and methods are provided for determining placement of content items, e.g., advertisements and/or images, on a rendered page relative to other content items on the page based upon bid value.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of commonly assigned application U.S.application Ser. No. 10/977,824, filed on Oct. 29, 2004, entitled“Systems and Methods for Determining Bid Value for Content Items to bePlaced on a Rendered Page”, which is hereby incorporated herein byreference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

TECHNICAL FIELD

The present invention relates to computing environments. Moreparticularly, the present invention relates to systems and methods fordetermining the value of bids placed by content providers for placementpositions on a rendered page, e.g., a web page. Additionally, thepresent invention relates to systems and methods for determining theplacement of content items, e.g., advertisements and/or images, on arendered page relative to other content items on the page.

BACKGROUND OF THE INVENTION

Searching and choosing products and services through computer-basedsearch engines has become increasingly prolific in recent years. Assuch, content providers, i.e., those companies and/or individualsdesiring content specific to their product(s) or service(s) to bedisplayed as a result of a given search engine query, e.g., advertisers,have begun to understand the value that placement of content items,e.g., descriptors or advertisements of their products or services, as aresult of a search engine query can have on their sales.

For instance, if a user accesses a hosting web site, e.g., a searchengine web site, and appropriately inputs a query for “Hawaiian hotels”,it is the user's desire to have a listing of hotels in Hawaii displayedas a result of the query. There are, however, numerous hotels in Hawaiiand the order in which descriptors of these hotels are displayed as aresult of the query can have a significant impact on whether or not theuser will access information associated with a particular hotel, e.g.,by selecting a particular content item or descriptor on the queryresults page which links the user to additional information about theassociated hotel. For example, a user is more likely to accessinformation associated with a hotel that is displayed in one of the topthree positions in a vertical listing of hotel links that is displayedas a result of the query than to access the information associated witha hotel that appears at the bottom of the display list. As such,determining the order in which the various content items or descriptorsappear in the displayed query results listing has become a task of greatinterest to search engine web sites and content providers alike, both ofwhom would like to maximize their revenue.

Typically, search engines permit content providers to bid for particularwords and/or phrases, such words and/or phrases being referred to hereinas “bid terms”, as a way for determining the order in which contentitems or descriptors which provide links to the content providers'information will be displayed. Bids are typically made as cost-per-click(CPC) commitments. That is, the content provider bids a dollar amount itis willing to pay each time a user selects or clicks on a displayedcontent item (e.g., an advertisement or image) as a result of the searchengine query and thus accesses the information associated therewith.

One method that search engines may use to determine placement ofdifferent content descriptors or advertisements is to simply rank by theCPC bid and give the best or most prominent placement to the contentprovider bidding the highest amount. For instance, Hotel A may “bid” oragree to pay the search engine $1.00 for each user that accesses itsinformation as a result of appearing in the search results of a givenquery while Hotel B may “bid” or agree to pay the search engine $1.50for each user that accesses its information upon appearing in the queryresults. In this instance, Hotel B would “win” the bid and, accordingly,its content descriptor would be placed in a more prominent position onthe web page on which the results of a search initiated by a query thatexactly or partially matches the bid terms are displayed. For instance,a descriptor of Hotel B may be displayed in the first position of aplurality of vertically aligned content item positions that aredisplayed as a result of the query.

One drawback to this approach, however, is that it does not take intoaccount the probability that a user will access the informationassociated with the advertisement (or other content) and, accordingly,all of the risk resides with the search engine. For instance, in anextreme example, a content provider may place an inordinately large bidfor an advertisement that has a click-through-rate (CTR) of zero. Suchscenario would result in the advertisement being placed prominently onthe web page as the result of an appropriate query with no charge to thecontent provider and, thus, no profit to the search engine.

Another method that search engines may use to determine the placement ofcontent items as the result of a particular query is to take the productof the content-provider's CPC bid and the probability that a user willaccess the information associated with the advertisement (or othercontent item) and provide the most prominent placement to the contentprovider having the highest product. In this way, the search engine mayminimize its risk with respect to the above-described scenario whereinclick-through probability was not taken into account and, accordingly,can attempt to maximize its expected profit. However, the probabilitythat a user will access the information associated with a particularcontent item or descriptor may be difficult to determine, particularlywhen the content provider lacks a history with the search engine thatcan be empirically evaluated.

Additionally, in either of the above-described placement scenarios,content providers are unable to view the bids that are being placed bytheir competitors. As such, the only way that they can determine howtheir bids compare to their competitors is by examining the positionthey are awarded on the search engine web site relative to other contentproviders as a result of an appropriate query.

Accordingly, a method for determining relative placement of contentitems as the result of a search engine query wherein risk to the searchengine may be minimized while its profit may be maximized would bedesirable. Further, it would be advantageous if the relative placementof content items could be altered as the search engine developed ahistory with the content providers that is capable of being empiricallyevaluated. Additionally, a system and method for permitting providers toview information regarding their bids relative to their competitorsprior to the ultimate placement being determined would be advantageous.

BRIEF SUMMARY OF THE INVENTION

The present invention relates to systems and method for determining theplacement of content items or descriptors, e.g., advertisements and/orimages, on a rendered page (e.g., a web page) relative to other contentitems on the page, such systems and methods being based upon bid value.Accordingly, in one aspect, the present invention is directed to acomputer-implemented method for determining bid values for content to beplaced on a rendered page according to a given context, e.g., the searchresults listing for a particular query initiated on a search engine website. The method includes receiving a plurality of bids each associatedwith a content item, at least one of the plurality of bids being basedupon a pricing model that is different from a pricing model upon whichat least one other of the plurality of bids is based, and determining abid value for each of the plurality of bids. For example, contemplate ascenario wherein two bids are received. In one scenario, each of thebids may be based upon a single pricing model, the pricing model for thefirst bid being different from the pricing model of the second bid. Inanother example, the first bid may be based upon a combination of morethan one pricing model while the second bid may be based upon a singlepricing model, whether the same as one of the pricing models on whichthe first bid is based or different therefrom.

In one pricing models, i.e., a cost-per-click (CPC) pricing model, a bidvalue may be determined as a CPC bid multiplied by the click-throughrate (CTR) of a content item (e.g., an advertisement or image). Inanother aspect, the present invention is directed to acomputer-implemented method for estimating CTR for content items to beplaced on a page (e.g., a web page) rendered in accordance with a givencontext, for instance, the search results for a particular queryinitiated on a search engine web site, the estimate being based, inpart, upon observed data.

In yet another aspect, the present invention is directed to acomputer-implemented method for determining relative placement ofcontent items based upon bid value, the content items to be placed on apage (e.g., a web page) rendered according to a given context. Themethod includes receiving a plurality of bids, each associated with acontent item, and determining relative placement of the content items onthe page according to Vickrey-like or modified Vickrey-like auctionprinciples.

In another embodiment, the present invention is directed to acomputer-implemented method for introducing randomness into the relativeplacement of content items based upon bid value, the content items to beplaced on a page (e.g., a web page) rendered according to a givencontext. In one embodiment of the present invention, randomness isintroduced by permitting each content provider placing a bid on aparticular placement position on the rendered page to have its contentitem allocated to that particular placement position at a frequencyproportional to the expected performance of its content item relative tothe expected performance of the content items of all content providersplacing bids for the same position.

In still another embodiment, the present invention is directed to acomputer-implemented method for determining bid values based upon one ormore targeting attributes for which a bidder (e.g., a content provider)may desire to place an incrementally different bid. The method permits abidder to allocate a “base bid” for a basic good and then incrementallyadjust the base bid for each attribute that it perceives as being ofincreased or decreased value. Subsequently, at the time an auction forthe good is run, the total for each bidder's base bid and all of theincremental values which hold true regarding the good's attributes maybe summed and a bid value determined.

In another embodiment, the present invention is directed to acomputer-implemented method for determining the relative placement ofcontent items of varying shapes and sizes based upon bid value, thecontent items to be placed on a page (e.g., a web page) renderedaccording to a given context. The method permits content providers tobid for mixed or bundled content items positions if, e.g., the contentitem desired to be placed on the rendered page does not fit within theparameters of a single defined content item position.

In yet another embodiment, the present invention is directed to acomputer-implemented method for estimating the relative placementposition of a particular content item based upon a proposed bid anddisplaying such position estimate to a content provider. Further, thepresent invention is directed to a computer-implemented method forestimating the value a content provider would have to bid to have a highlikelihood of having its content item placed in a desired content itemposition and displaying the bid value/pricing estimate to the contentprovider.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The present invention is described in detail below with reference to theattached drawing figures, wherein:

FIG. 1 is a block diagram of an exemplary computing environment suitablefor use in implementing the present invention;

FIG. 2 is a flow diagram showing a computer-implemented method fordetermining bid values for content items (e.g., advertisements orimages) in accordance with an embodiment of the present invention;

FIGS. 3A and 3B are flow diagrams showing computer-implemented methodsfor normalizing and estimating, respectively, click-through-rates (CTR)in accordance with embodiments of the present invention;

FIG. 4 is a flow diagram showing a computer-implemented method for usingVickrey-like auction principles to determine relative placement ofcontent items on a rendered page in accordance with an embodiment of thepresent invention;

FIG. 5 is a flow diagram showing a computer-implemented method fordetermining the price a winning bidder must pay for being awarded anitem using Vickrey-like auction principles in accordance with anembodiment of the present invention;

FIG. 6 is a flow diagram showing a computer-implemented method forintroducing randomness into the relative placement allocation of contentitems on a rendered page in accordance with an embodiment of the presentinvention;

FIG. 7 is a flow diagram showing a computer-implemented method fordetermining bid values based upon one or more targeting attributes inaccordance with an embodiment of the present invention;

FIG. 8A is an illustrative screen display of an exemplary user interface(UI) wherein a search phrase (or bid term) and base bid may be indicatedand displayed;

FIG. 8B is an illustrative screen display of an exemplary UI whereintargeting based upon user location may be indicated and displayed;

FIG. 8C is an illustrative screen display of an exemplary UI whereintargeting based upon the day of the week on which a search is beingconducted may be indicated and displayed;

FIG. 8D is an illustrative screen display of an exemplary UI whereintargeting based upon the time of day on which a search is beingconducted may be indicated and displayed;

FIG. 8E is an illustrative screen display of an exemplary UI whereintargeting based upon the awarded placement position may be indicated anddisplayed;

FIG. 9 is a flow diagram showing a computer-implemented method fordetermining relative placement of content items of varying shapes and/orsizes in accordance with an embodiment of the present invention;

FIG. 10 is a schematic diagram showing an example in which a mixed orbundled auction methodology may be implemented in accordance with anembodiment of the present invention;

FIG. 11 is a flow diagram showing a computer-implemented method forestimating the position on a rendered page where a content item islikely to appear in accordance with an embodiment of the presentinvention;

FIG. 12 is an illustrative screen display of an exemplary user interface(UI) wherein a base bid may be entered or selected by a bidder anddisplayed in accordance with an embodiment of the present invention;

FIG. 13 is an illustrative screen display of an exemplary user interface(UI) having an expanded advanced price calculator options area relativeto the screen display of FIG. 12;

FIG. 14 is a flow diagram showing a computer-implemented method fordetermining a position estimate based upon a determined bid value inaccordance with an embodiment of the present invention;

FIG. 15 is an illustrative screen display of an exemplary user interface(UI) wherein desired position and frequency information may be enteredor selected by a bidder and displayed in accordance with an embodimentof the present invention;

FIG. 16 is an illustrative screen display of an exemplary user interface(UI) having an expanded advanced price calculator options area relativeto the screen display of FIG. 15;

FIG. 17 is a flow diagram showing a computer-implemented method forestimating the value a content provider would have to bid to have a highlikelihood of having its content item placed in a desired content itemposition in accordance with an embodiment of the present invention;

FIG. 18 is a flow diagram showing a computer-implemented method fordetermining a bid value/pricing estimate based upon a desired contentitem position and frequency in accordance with an embodiment of thepresent invention; and

FIG. 19 is a flow diagram showing a computer-implemented method forselecting content items to be placed on a rendered page, the contentitems having varying shapes and/or sizes, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides systems and methods for determining theplacement of content items or descriptors, e.g., advertisements and/orimages, on a rendered page (e.g., a web page) relative to other contentitems on the page. Additionally, the present invention provides systemsand methods for determining relative placement of content items on arendered page based upon bid value, the bid values being determinedbased upon a plurality of pricing models. The present invention furtherprovides systems and methods for estimating click-through-rates forcontent items to be placed on a rendered page based, in part, uponobserved data. Still further, the present invention provides systems andmethods for introducing randomness into the relative placement ofcontent items based upon bid value. Additionally, the present inventionprovides systems and methods for determining bid values based upon oneor more targeting attributes for which a bidder may desire to place anincrementally different bid. Further, the present invention providessystems and methods for determining the relative placement of contentitems of varying shapes and sizes based upon bid value. The presentinvention further provides systems and methods for estimating therelative placement position of a particular content item based upon aproposed bid and displaying such position estimate to a contentprovider. Additionally, the present invention provides systems andmethods for estimating the value a content provider would have to bid tohave a high likelihood of having its content item placed in a desiredcontent item position and displaying the bid value/pricing estimate tothe content provider.

Having briefly described an overview of various embodiments of thepresent invention, an exemplary operating environment for the presentinvention is described below.

Exemplary Operating Environment

Referring to the drawings in general and initially to FIG. 1 inparticular, wherein like reference numerals identify like components inthe various figures, an exemplary operating environment for implementingthe present invention is shown and designated generally as computingsystem environment 100. The computing system environment 100 is only oneexample of a suitable computing environment and is not intended tosuggest any limitation as to the scope of use or functionality of theinvention. Neither should the computing environment 100 be interpretedas having any dependency or requirement relating to any one orcombination of components illustrated in the exemplary operatingenvironment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc., that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

With reference to FIG. 1, an exemplary system for implementing thepresent invention includes a general purpose computing device in theform of a computer 110. Components of computer 110 may include, but arenot limited to, a processing unit 120, a system memory 130, and a systembus 121 that couples various system components including the systemmemory to the processing unit 120. The system bus 121 may be any ofseveral types of bus structures including a memory bus or memorycontroller, a peripheral bus, and a local bus using any of a variety ofbus architectures. By way of example, and not limitation, sucharchitectures include Industry Standard Architecture (ISA) bus, MicroChannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnect (PCI) bus also known as Mezzanine bus.

Computer 110 typically includes a variety of computer-readable media.Computer-readable media can be any available media that can be accessedby computer 110 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such ascomputer-readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to, RAM,ROM, EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by computer 110. Communication media typicallyembodies computer-readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer-readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system (BIOS)133, containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system 134, applicationprograms 135, other program modules 136, and program data 137.

The computer 10 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks (DVDs), digital video tape, solid state RAM,solid state ROM, and the like. The hard disk drive 141 is typicallyconnected to the system bus 121 through a non-removable memory interfacesuch as interface 140, and magnetic disk drive 151 and optical diskdrive 155 are typically connected to the system bus 121 by a removablememory interface, such as interface 150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 1, provide storage of computer-readableinstructions, data structures, program modules and other data for thecomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing operating system 144, application programs 145, other programmodules 146, and program data 147. Note that these components can eitherbe the same as or different from operating system 134, applicationprograms 135, other program modules 136, and program data 137. Operatingsystem 144, application programs 145, other programs 146 and programdata 147 are given different numbers herein to illustrate that, at aminimum, they are different copies. A user may enter commands andinformation into the computer 110 through input devices such as akeyboard 162 and pointing device 161, commonly referred to as a mouse,trackball or touch pad. Other input devices (not shown) may include amicrophone, joystick, game pad, satellite dish, scanner, or the like.These and other input devices are often connected to the processing unit120 through a user input interface 160 that is coupled to the systembus, but may be connected by other interface and bus structures, such asa parallel port, game port or a universal serial bus (USB). A monitor191 or other type of display device is also connected to the system bus121 via an interface, such as a video interface 190. In addition to themonitor 191, computers may also include other peripheral output devicessuch as speakers 197 and a printer 196, which may be connected throughan output peripheral interface 195.

The computer 110 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer180. The remote computer 180 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 110, although only a memory storage device 181 has beenillustrated in FIG. 1. The logical connections depicted in FIG. 1include a local area network (LAN) 171 and a wide area network (WAN)173, but may also include other networks. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet.

When used in a LAN networking environment, the computer 110 is connectedto the LAN 171 through a network interface or adapter 170. When used ina WAN networking environment, the computer 110 typically includes amodem 172 or other means for establishing communications over the WAN173, such as the Internet. The modem 172, which may be internal orexternal, may be connected to the system bus 121 via the networkinterface 170, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 110, orportions thereof, may be stored in a remote memory storage device. Byway of example, and not limitation, FIG. 1 illustrates remoteapplication programs 185 as residing on memory device 181. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

Although many other internal components of the computer 110 are notshown, those of ordinary skill in the art will appreciate that suchcomponents and the interconnection are well known. Accordingly,additional details concerning the internal construction of the computer110 need not be disclosed in connection with the present invention.

When the computer 110 is turned on or reset, the BIOS 133, which isstored in the ROM 131, instructs the processing unit 120 to load theoperating system, or necessary portion thereof, from the hard disk drive141 into the RAM 132. Once the copied portion of the operating system,designated as operating system 144, is loaded into the RAM 132, theprocessing unit 120 executes the operating system code and causes thevisual elements associated with the user interface of the operatingsystem 134 to be displayed on the monitor 191. Typically, when anapplication program 145 is opened by a user, the program code andrelevant data are read from the hard disk drive 141 and the necessaryportions are copied into RAM 132, the copied portion represented hereinby reference numeral 135.

Method for Determining Bid Values Based Upon a Plurality of PricingModels

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

As previously mentioned, the present invention relates to systems andmethods for determining the value of bids placed by content providersfor placement positions on a rendered page, e.g., a web page. As bidvalue may be measured in terms of dollars paid to the hosting web site,e.g., a search engine web site, there are various different pricingmodels that can be mixed together in a principled manner to rank contentitems according to value.

One such pricing model is referred to herein as a “cost-per-impression”pricing model or a CPI model (also referred to as a “cost-per-thousand”or CPM model). In the CPI pricing model, a content provider, e.g., anadvertiser, pays the search engine a fixed dollar amount each and everytime its content item, e.g., its advertisement or image, is displayed asthe result of an appropriate search engine query. The advertisers' bidsin this pricing model are equivalent to their value and, accordingly,there is no uncertainty involved in calculating the bid value.

A second pricing model is referred to herein as a “cost-per-click”pricing model or a CPC model. In the CPC model, the content providerpays the search engine a fixed dollar amount each time a user accessesthe information associated with the content item displayed as the resultof an appropriate search engine query. That is, as content itemstypically represent selectable links to more detailed information aboutthe subject matter thereof, in the CPC model, the content provider paysthe search engine only when a user selects or clicks on its contentitem, thus accessing the associated information. The bid value in thispricing model is the CPC bid multiplied by the click-through-rate (CTR)of the content item (e.g., the advertisement or image).Click-through-rate is an unknown value and, accordingly, it must beestimated. Various methods for estimating CTR may be utilized, a numberof which are discussed hereinbelow.

The uncertainty in this scenario rests on the search engine because ifthe content item is displayed in a prominent position as the result ofan appropriate query but no users access the information associated withthe item, the search engine receives no payment from the contentprovider.

A third pricing model is referred to herein as a “cost-per-sale” pricingmodel or a CPS model. In the CPS model, the content provider bids afixed dollar amount it is willing to pay the search engine for each salethat results from a user being shown its content item as the result ofan appropriate search engine query. In this scenario, the search enginereceives payment from the content provider only if a user purchases theproduct or service indicated by the content item and not if someonemerely accesses the information associated with the content item butdoes not complete the sale. The bid value in this case is the CPS bidmultiplied by the expected probability that showing the advertisementwill result in a sale. The expected sales probability must be estimated,the ease with which this may be done being based largely upon thehistory between the content provider and the search engine.

A fourth pricing model is referred to herein as a “revenue sharing”model. With revenue sharing, the content provider bids a percentage ofthe sales price that they are willing to pay to the search engine if auser buys anything as a result of being shown the content item in theresults of an appropriate search engine query, whether or not theproduct or service sold is identical to the product or serviceassociated with the content item. The bid value is calculated in thisinstance as the revenue sharing percentage bid, e.g., 5%, multiplied bythe expected sales price. There is some tracking involved in the revenuesharing pricing model, the ease with which such tracking may be donebeing based upon the history between the content provider and the searchengine, as well as the breadth of products and/or services offered bythe content provider.

The above-described pricing models all have one feature in common: theycan be evaluated based upon a dollar figure. That is, the search enginemay determine a bid value for each content provider which maysubsequently be compared with the bid values determined for competingcontent providers, regardless of which pricing model each contentprovider utilized in formulating its bid. For instance, if contentprovider A provides the search engine with a CPI bid and contentprovider B provides the search engine with a CPS bid, the search enginecan determine the bid value for each of the bids received and, thus,appropriately compare the bids. In this way, the search engine can moreadequately maximize its expected revenue. Additionally, as the searchengine's expected revenue can be estimated and/or determined as a dollaramount, combinations of pricing models with respect to a single contentprovider may also be utilized, as more fully described below.

With reference to FIG. 2, a flow diagram is illustrated which shows acomputer-implemented method 200 for determining bid values for contentitems (e.g., advertisements or images) in accordance with an embodimentof the present invention. The content items in method 200 may be placedon a page (e.g., a web page) rendered according to a given context, forinstance, the search results for a particular query initiated on asearch engine web site. Initially, as shown at block 210, the systemreceives a plurality of bids each associated with a content item, atleast one of the plurality of bids being based upon a pricing model thatis different from the pricing model upon which at least one other of theplurality of bids is based. These bids may be, for instance, based uponCPI bids, CPC bids, CPS bids, or revenue sharing bids, each of which washereinabove described. It will be understood by those of ordinary skillin the art, however, that bids may be placed based upon a combination ofpricing models and such combinations are intended to be included in thedefinition of such term. For instance, a bidder may place a bid for$0.05 per impression and $0.10 per occasion that an impression isclicked through. Such variations are intended to be within the scope ofthe present invention.

Depending upon the type of bid received, the system subsequentlydetermines a bid value for each bid received, as shown at block 212. Aseach of the pricing models excepting the CPI pricing model includes sometype of estimated probability in determining the bid value, in oneembodiment, the system may only accept CPI bids from a content provideruntil enough user access data can be collected to determine a reliableCTR estimate, as more fully described below. Once a reliable CTRestimate may be determined, the content provider may “graduate” suchthat the system accepts CPC bids from it. Similarly, as more user accessdata is collected so that it may more confidently be predicted what theexpected sales rate and/or the expected sales revenue for a givencontent provider may be, the content provider may “graduate” such thatthe system accepts CPS and/or revenue sharing bids from the contentprovider. All such variations are contemplated to be within the scope ofthe present invention.

In another embodiment, the system may accept non-CPI bids from allcontent providers but adjust the bid according to risk. One method foradjusting the bid is to add a “risk penalty” to the bids placed bycontent providers with which the search engine has little or no history.The more uncertain the search engine is about a CTR or sale probability,the larger the risk penalty. The risk penalty may be used to compute aneffective CTR, which may then be used in determining expected bid value.

One risk penalty methodology which may be utilized determines aneffective probability equal to the expected or estimated probabilityless an amount proportional to the variance of the estimate of theprobability. The variance of the estimate of the probability, in turn,may be estimated via one of a set of standard probability models, e.g.,the Dirichlet model, as known to those of ordinary skill in the art.Note that when no data is available to estimate the probabilities, theeffective probability can go below zero, in which case a CPC bid may beignored.

By way of example only, suppose a content provider desires to place aCPC bid and that a Beta(1,1) prior is used for the probability of a userclicking on the content item associated therewith. After the systemevaluates user behavior and determines c clicks (i.e., c occasionswherein a user accesses the information associated with the contentitem) and n non-clicks (i.e., n occasions wherein a user does not accessthe information associated with the content item), the posteriorprobability of a user clicking is a Beta(1+c, 1+n) distribution. Theexpected click rate (i.e., the expected value of the Beta distribution)is accordingly:E(rate)=(1+c)/(2+c+n).

The variance of the click rate (i.e., the variance of the Betadistribution) isVar(rate)=(1+c)(1+n)/[(2+c+n)²(3+c+n)].

The effective click-through rate (CTR) may then be determined asP _(click) =E(rate)−k×Var(rate)

for some value of k that determines how risk averse the content provideris. In practice, a prior may be selected that adequately reflects CTRs,for example, a Beta prior with small sample size (e.g., 1) and a meanequal to the overall CTR or sales rate of the content provider site. Therisk-adjusted expected revenue for a bid and, in turn, the bid value, isthe CPC bid price multiplied by the effective click-through rate,P_(click).

Similarly, an effective click-through to sale, P_(sale), may bedetermined and utilized for computation of the risk-adjusted expectedrevenue for CPS bids.

It should be noted that the above-described risk adjustment methodologyis intended to be exemplary only. Any number of risk adjustmentmethodologies may be utilized, as known to those of ordinary skill inthe art, and all such methodologies are intended to be within the scopeof the present invention. For instance, risk adjustment could becalculated based upon the probability that the content provider willdefault on their payment, or the like.

In another embodiment, risk may be adjusted based upon a “risk bonus” tofavor those content providers for whom CTRs are not well known, that is,those content providers with whom the search engine has limited history.In this instance,P _(click) =E(rate)+k×Var(rate).

Both the risk penalty and risk bonus embodiments of the presentinvention may be augmented to include a bid floor for the minimum bidfor showing a content item using the expected revenue in the simple caseand using risk adjusted expected revenue in the second case.Additionally, bid terms may be accepted which include locality specificterms, including, but not limited to, zip code or the like, as morefully described below. All such variations are contemplated to be withinthe scope hereof.

Referring back to FIG. 2, once the bid values have been determined, oneor more of the content items associated with the plurality of bids maybe selected based upon the respective determined bid values, forinstance, for placement in a prominent position on the rendered page.This is shown at block 214. Typically, the content item having thehighest bid value will be awarded the most prominent position upondisplay. For instance, contemplate a scenario wherein the positionsavailable for placing content items are ordered as pos₁, pos₂, pos₃, . .. pos_(k), with pos₁ being the top link of a vertical listing of linkson a search engine web page that is displayed as the result of aparticular user query, pos₂ being the second link, and so on. In thisinstance, the content item having the highest bid value may be awardedpos₁.

Subsequently, the one or more selected content items may be displayed,as indicated at block 216. Alternatively or in conjunction, as shown atblock 218, a ranking for the content items associated with the pluralityof bids may be determined based upon the respective determined bidvalues. The selected content items may subsequently be displayedaccording to the determined ranking, as indicated at block 220.

Next, as indicated at block 222, the system may receive informationabout a user, e.g., demographic and/or gender-related information andthe context may be determined based upon the user information received.This is indicated at block 224. Systems and methods for utilizing userinformation to determine bid value are more fully described below.

Methods for Estimating Click-Through Rates of Content Items on aRendered Page

As previously discussed, one pricing model under which bid values may bedetermined is a cost-per-click pricing model or CPC pricing model. A bidvalue under this pricing model may be determined as the CPC bidmultiplied by the click-through-rate (CTR) of the content item (e.g.,the advertisement or image). Click-through-rate is an unknown value and,accordingly it must be estimated. As one of ordinary skill in the artwill appreciate, appropriately estimating click-through rate (CTR) isimportant to ensure appropriate bid value calculations and, e.g., theresultant relative content item placement on a rendered page.Accordingly, the present invention further relates to systems andmethods for estimating CTR for content items to be placed on a page(e.g., a web page) rendered according to a given context, for instance,the search results for a particular query initiated on a search engineweb site.

Certain content item positions on a rendered page (e.g., a web page)have higher CTRs than other content item positions. For example, a useris more likely to access the information associated with a content itemthat is displayed in one of the top three positions in a verticallisting of content items (i.e., a user is more likely to “click-through”on a content item that is displayed in one of the top three positions ina vertical listing of content items) than to access the informationassociated with a content item that appears at the bottom of the displaylist. As such, in order to appropriately compare CPC bids for a contentitem that may appear in any one of a plurality of content item positionson the rendered page, historical CTR data must be adjusted such that theCTR for a more prominent or desired position is comparable to the CTRfor a less prominent or desired position. Otherwise, content providershaving better placement on the rendered page will be initially unfairlyfavored.

In adjusting or “normalizing” CTR, both the position of the content itemon the rendered page as well as its content item type (e.g., mainline orsidebar) must be considered. The combination of these two variables isreferred to herein as “context”. Referring now to FIG. 3A, acomputer-implemented method for normalizing click-through-rate such thatcontext may be eliminated as a variable in CPC bid comparisons is shownand designated generally as reference numeral 300. Initially, asindicated at block 310, a particular context may be selected as areference context (r), the reference context being associated with aparticular content item position/content item type combination. Theselected reference context need not have the highest CTR of the contentitems on the rendered page, however, it is currently preferred that thereference context be one of the contexts with a high CTR (e.g., aposition 1 side-bar) in order to ensure good CTR estimates.

Subsequently, as indicated at block 312, the CTR of the referencecontext (r) may be determined based upon observed data (CTR_(r)). TheCTR of each context on the rendered page (represented herein as adynamic context (j)) relative to the CTR of the reference context (i.e.,CTR_(r)) may then be determined and delineated as R_(j). This isindicated at block 314.

For example, contemplate that it is desired to determine the CTR of aparticular context (i) relative to the reference context and that thereference context, in this example, has a CTR represented by CTR₁ andgiven a value equal to one. R_(i) (that is, the relative CTR ofparticular context (i) relative to CTR₁) may be represented asCTR_(i)/CTR₁. Thus, a content item is one-half as likely to be clickedin particular context (i) than in the reference context, R_(i)=½. (Notethat R₁ (that is the CTR of the reference context (CTR₁) relative to theCTR of the reference context (CTR₁) will always be equal to one.)

In one embodiment, the range of permitted values for R_(i) is [ 1/100,20]. That is, if the maximum value is exceeded, then R_(i) may be set tobe equal to 20. Similarly, if R_(i) is less than the minimum value, thenit may be set to be equal to 1/100. If the reference context is selectedappropriately (i.e., is selected as one of the contexts on the renderedpage having one of the highest CTRs), then R_(i) should not reach valuesthat are very large and certainly should not approach the maximum value.

Returning to FIG. 3A, the actual number of impressions of a particularcontent item (k) in context (j) that are clicked through subsequentlymay be determined based on observed data and designated as Mk_(c=1)(j).This is indicated at block 316. Additionally, as indicated at block 318,the actual number of impressions of the given content item (k) incontext (j) that are not clicked through may also be determined basedupon observed data and designated as Mk_(c=0)(j)

Subsequently, as indicated at block 320, the number of impressions ofthe given content item (k) that are clicked through may be normalized toany context on the rendered page, the normalized number of impressionsbe designated as Mk_(c=1). Mk_(c=1) may be determined based upon thefollowing formula:${Mk}_{C = 1} = {\sum\limits_{i}^{\quad}\quad\frac{{Mk}_{C = 1}(j)}{R_{j}}}$

Next, as indicated at block 322, the number of impressions of the givencontent item (k) that are not clicked through may be normalized to anycontext on the rendered page, the normalized number of impressions beingdesignated as Mk_(c=0). Mk_(c=0) may be determined based upon thefollowing formula:${Mk}_{C = 0} = {\sum\limits_{j}^{\quad}\quad{{Mk}_{C = 0}(j)}}$

(Note: Mk, the actual number of impressions normalized to any context onthe rendered page, is equal to Mk_(c=0)+Mk_(c=1).)

Subsequently, as indicated at block 324, it is determined whether or notthe bid (or bids) for which it is desired that CTR be normalized aretargeted or non-targeted bids. (Methods and systems for targeting bidsbased upon various attributes and attribute options are more fullydescribed hereinbelow.) If it is determined that the bid for which it isdesired that CTR be normalized is a non-targeted bid, a single CTRestimate for content item (k) may be determined based upon the followingformula:${p_{k}({click})} = {\frac{{Mk}_{C = 1}}{{Mk}_{C = 1} + {Mk}_{C = 0}}.}$This is indicated at block 326. Click-through-rate may be stored atcontent item level thus enabling optimization of content items forranking and display.

If the bid (or bids) for which it is desired that CTR be normalized aretargeted bids, however, multiple estimates for CTR may be determined,one CTR for each required combination of targeting features. This isindicated at block 328. Being able to adjust CTR by targeting is usefulas it makes CTRs based upon content items having different targetingfeatures comparable. For instance, a CTR under a broad match query islikely to be lower than the CTR under an exact match query. If CTR isadjusted, broad match content providers may potentially be forced to payhigher CPC because of their lower CTR.

Determining an estimated CTR for bids based on one or more targetedfeatures is more complicated than with non-targeted bids as it requiresknowledge of a number of items that are not necessary for anon-targeting bid. First, the impact of each targeting feature on theCTR must be known. For instance, for a given content item, how much morelikely is a New York user to click on it than a Seattle user? Secondly,the adjusted CTR must be dynamically calculated at run-time accordingto, in part, the search user's profile. For example, when it is knownthat the user is a New York user, the CTR must be adjusted with the NewYork effect at run-time, whereas if the search user is not from NewYork, the CTR may be adjusted with the New York effect set to zero.

For at least these reasons, an offline model is necessary to answer thefirst inquiry (i.e., the impact of each targeting feature on the CTR)and a real-time formula is necessary to answer the second inquiry (i.e.,the search user profile).

The offline model makes sure that the impact of every possible value ofthe targeting attributes on CTR is known. That is, it will be known, forinstance, that a female user is x % more likely to click on a particularcontent item, or a New York user is y % less likely to click on acontent item than a user from any other demographic. Each possible valueof the targeting attributes is referred to herein as a “feature”.Therefore, Location=New York is a feature, as is Gender=Female.

Referring now to FIG. 3B, a computer-implemented method for estimatingclick-through-rate when one or more bids is a targeted bid isillustrated. Initially, as indicated at block 330, the system estimatesa CTR for content item that is independent of any targeted attribute.This estimate is determined by the same formula as the P_(k)(Click)under the non-targeting model discussed hereinabove.

Subsequently, as indicated at block 332, an attribute is determined atthe listing level using the following formula:$a = {\prod\limits_{i}^{\quad}\quad\frac{p\left( {A_{i} = {{default}_{i}\text{❘}{click}}} \right)}{p\left( {A_{i} = {{default}_{i}\text{❘}{noclick}}} \right)}}$

wherein default_(i) is the default value for attribute i,P(A_(i)=default_(i)|click) is the percentage of all impressions that areclicked through having default_(i) (including those for which attributei is unknown), P(A_(i)=default_(i)|noclick) is the percentage of allimpressions that are not clicked through having the default_(i)(including those for which attribute i is unknown), and the product istaken over all attributes that have at least one targeted value in theresults listing.

Next, as indicated at block 334, a parameter (a) for each possible valuea of a targeted attribute A (i.e., a feature) is determined using thefollowing formula:$\lambda_{a} = {\frac{p\left( {A = {a\text{❘}{click}}} \right)}{p\left( {A = {a\text{❘}{noclick}}} \right)}\frac{p\left( {A = {{default}\text{❘}{noclick}}} \right)}{p\left( {A = {{default}\text{❘}{click}}} \right)}}$

wherein the default value for attribute a is default_(a), P(A=a|click)is the percentage of impressions having attribute A=a that are clickedthrough, P(A=a|noclick) is the percentage of all impressions havingattribute A=a that are not clicked through, P(A=default|click) is thepercentage of all impressions having default_(a) that are clickedthrough (including those for which attribute A is unknown), andP(A=default|noclick) is the percentage of all impressions havingdefault_(a) that are not clicked through (including those for whichattribute A is unknown).

To extract each parameter_(i), contemplate that feature i corresponds toattribute A having value a. (Note that there will never be a feature orcorresponding to the default value of an attribute.)

Subsequently, as shown at block 336, the targeted CTR is determined forcontent item (k) for each combination of features using the followingformula:${p\left( {{Click}\quad{on}\quad{ad}\quad k\text{❘}{Features}} \right)} = \frac{\mu_{k}a{\prod\limits_{i}^{\quad}\quad{\lambda_{i}^{f}i}}}{1 + {\mu_{k}a{\prod\limits_{i}^{\quad}\quad{\lambda_{i}^{f}i}}}}$

where $\mu_{k} = {\frac{p_{k}({click})}{1 - {p_{k}({click})}}.}$The input feature, ƒ_(i), is a binary function that is equal to one if agiven user has the attribute and value associated with parameter_(i) andzero otherwise. The formula has one parameter_(i) for each input featureƒ_(i).

Initial values for the offline parameters for both the targeting andnon-targeting models must be determined. In all casesp(A=i|noclick)=P(A=i) and p(A=i|click)=p(A=i) may be initialized. Thus,all that remains is to find a good estimate for p_(k)(click). Using thesame initial value for all content items in the results listing (i.e.,p_(k)(click)=p(click) for all content items (k)), three cases areconsidered.

First, contemplate a scenario wherein content item (k) is for a newquery term. That is, no content providers have previously bid on theterm. In this scenario, p(click) may be set to the global click rate.Note that a bad estimate in this case is not a big deal as in alllikelihood, there will not be any competition for this particular queryterm and, thus, a bid floor will likely be charged. The only way inwhich there may be competition for this particular query term is if twocontent providers both bid on the new query term at the same time. Inthis case, the CTR estimates will be the same for both content providersand the content provider providing the highest bid (which will also bethe bid having the highest bid value) will be awarded the position beingauctioned. Additionally, the winning content provider will be chargedthe bid provided by the competing content provider in accordance withVickrey-type auction principles, as more fully described below.

Second, contemplate a scenario wherein the content item is for an oldquery term but the content provider is new. In this case, the medianp(click) for the content providers who currently have content itemsdisplayed for the bid term will be used for the new content provider'sestimated CTR.

Third, contemplate a scenario wherein the content item is for an oldquery term but the content provider has content items running for otherquery terms as well. This case is similar to the new content providerscenario, except that instead of using the median p(click), the relativeCTRs for this particular content provider's other queries will be usedto estimate CTR. If the content provider does better than average onother queries, it will be assumed that it will do better than average onthis query as well.

In this scenario, the percentage of other content items for which thiscontent provider has a p(click) greater than the median p(click) for thecorresponding query term is first determined. If this percentage, K, isless than 50%, the initial probability will be set as in the new contentprovider scenario described hereinabove. If, however, K is greater than50%, the initial probability will be set at the Kth percentileprobability of current content providers. That is, if K=80% and thereare 100 content items displayed in the query search results listing, thep(click) values will be sorted in descending order and the 20^(th)p(click) will be used. If this p(click) is greater than thesecond-highest p(click), however, the estimated CTR will be re-set to beequal to the second highest. This makes sure that a content providernever has the highest estimated p(click) until the system has actuallystarted keeping track.

A batch updating method may be used to update the offline parameters forboth the targeting and non-targeting models. In particular, after adata-gathering period, the parameters may be modified to better reflectthe observed fractions in the data. The update is a function of countsin the observed data. That is, because p_(k)(click) is calculated percontent item, impressions that are clicked through that are specific toa particular content item must be counted.

Let Mk_(C=1) be the number of effective impressions of content item (k)in the data that are clicked-through impressions and let Mk_(C=0) be thenumber of effective impressions of content item (k) in the data that areimpressions that are not clicked through. Let Mk=Mk_(C=0)+Mk_(C=1).Effective counts rather than actual counts are used to adjust fordifferences in click-through-rates for different contexts (i.e.,positions and render types). The specific method for computing effectivecounts was described hereinabove.

The update formula is as follows:${p_{k}^{\prime}({click})} = {{\beta^{Mk} \times {p_{k}({click})}} + {\left( {1 - \beta^{Mk}} \right)\left( \frac{{Mk}_{C = 1} + q}{{Mk}_{C = 1} + {Mk}_{C = 0} + s} \right)}}$

where q and s are constants with q<s (e.g., q=p(click), the globalprobability of any content item on the rendered page being clicked, ands=1). The denominator and numerator of the second part of the formulaadd only a small number (1 and q) such that P′_(k) (click) is not equalto zero.

Because the conditional probabilities are shared across all contentitems in a search results listing, impressions and click-throughs arecounted for the listing without regard to the content item itself. Inthis way, p (A=a|click) may be updated.

Let N be the total number of (actual) impressions of all content itemson the results listing. Let N_(C=1) be the number of impressions in thedata that are clicked through, and let N_(C=0) be the number ofimpressions in the data that are not clicked through. Similarly, letN_(A) _(i) _(=j,C=1) be the number of impressions where attribute i hasvalue j and there is click through, and let N_(A) _(i) _(=j,C=0) be thenumber of impressions where attribute i has value j and there is noclick through. The parameters subsequently may be updated as follows:${p^{\prime}\left( {A_{i} = {j\text{❘}{click}}} \right)} = {{\gamma^{N_{C = 1}} \times {p\left( {A_{i} = {j\text{❘}{click}}} \right)}} + {\left( {1 - \gamma^{N_{C = 1}}} \right)\left( \frac{N_{{A_{i} = j},{C = 1}} + w_{ij}}{N_{C = 1} + x_{ij}} \right)}}$and${p^{\prime}\left( {A_{i} = {j\text{❘}{noclick}}} \right)} = {{\gamma^{N_{C = 0}} \times {p\left( {A_{i} = {j\text{❘}{noclick}}} \right)}} + {\left( {1 - \gamma^{N_{C = 0}}} \right)\left( \frac{N_{{A_{i} = j},{C = 0}} + w_{ij}}{N_{C = 0} + x_{ij}} \right)}}$

where w_(ij) and x_(ij) are constants with w_(ij)<x_(ij) (e.g.,w_(ij)=p(A_(i)=j), the global probability that attribute i has value j,and x_(ij)=1.

In a currently preferred embodiment, =0.999 and =0.99 as these valueswork well for the on-line version of the update (i.e., when N=1).

Method for Determining Relative Placement of Content Items UsingVickrey-Type Auction Principles

The present invention further relates to systems and methods fordetermining relative placement of content items based upon bid value,the content items to be placed on a page (e.g., a web page) renderedaccording to a given context, for instance, the results of a particularquery run by a search engine. In one embodiment, the relative placementof the content items may be determined based upon bid values accordingto Vickrey-like auction principles. Vickrey-like auction principles arewell known to those of ordinary skill in the art and, accordingly, arediscussed only briefly herein. More detailed information aboutVickrey-like auctions may be found in Milgrom, P., Putting AuctionTheory to Work (Churchill Lectures in Economics); (Cambridge UniversityPress; Jan. 12, 2004); ISBN 0521536723, the entirety of which isincorporated by reference herein.

In a standard Vickrey-like auction, there are a number of itemsavailable for auction and a number of bidders who desire to bid on oneor more of the items. The bids for each item are received and theauction items are then allocated to the bidders. In one embodiment, thepresent invention relates to applying Vickrey-like auction principles todetermine relative placement of content items on a rendered page.

With reference to FIG. 4, a flow diagram is illustrated which shows acomputer-implemented method 400 for using Vickrey-like auctionprinciples to determine relative placement of content items on arendered page in accordance with an embodiment of the present invention.The content items in method 400 may be placed on a page (e.g., a webpage) rendered according to a given context, for instance, the searchresults listing for a particular query initiated on a search engine website.

Initially, as shown at block 410, the system receives a plurality ofbids for a particular item, e.g., a particular content item position,available for auction, each bid being associated with a content item.For instance, recall the scenario described hereinabove wherein aplurality of content providers are bidding for placement of theircontent on a rendered page relative to the content of other contentproviders, all such content falling within the given context. The“items” available for auction in this scenario may be thought of thevarious positions on the results page of a particular query run by asearch engine. By way of example only, the positions available to thecontent providers may be ordered as pos₁, pos₂, . . . , pos_(k), withpos₁ being the top-most position on the rendered page, pos₂ being thesecond position from the top of the rendered page, and so on.Accordingly, bids may be received from a number of content providers foreach position and, regardless of the pricing model under which a bid wasprovided, a bid value may be determined for each of the bids for eachposition. This is shown at block 412.

Subsequently, it may be determined which bidder is to be awarded theitem for which the bids were placed, i.e., the winning bidder, asindicated at block 414. For instance, the positions on the page may thenbe awarded to the content providers such that the total value ismaximized.

By way of example, suppose there are three positions on a rendered pageto be allocated and five content providers (A, B, C, D, and E) placebids on the positions. Irrespective of the type of bid received (e.g., aCPI bid, a CPC bid, a CPS bid, or a revenue sharing bid), suppose thebid values for pos₁ for the content providers are as follows: contentprovider A's bid has a bid value of $1.00, content provider B's bid hasa bid value of $0.90, content provider C's bid has a bid value of $0.80,content provider D's bid has a bid value of $0.75, and content providerE's bid has a bid value of $0.60. As content provider A's bid has thehighest bid value, content provider A will be awarded pos₁. All otherpositions on the rendered page will be similarly allocated among thosecontent providers bidding therefore, keeping in mind that in someinstances, there will be constraints within the auction that need to besatisfied. For example, each bidder may be permitted to be awarded onlya single item in the auction. Suppose in this example that contentproviders B and C are awarded the other two positions available forauction on the rendered page.

Once the allocation of items is determined, the price the winning biddermust pay for the item awarded is determined, as indicated at block 416.In a Vickrey-like auctions, the winning bidder typically pays for anitem the value that the bidder took away from the other winning biddersas a result of their participation in the auction. With reference toFIG. 5, a computer-implemented method for determining the price thewinning bidder must pay for the awarded item using Vickrey-like auctionprinciples in accordance with an embodiment of the present invention isillustrated and designated generally as reference numeral 500.Initially, as shown at block 510, the system determines the identify ofall bidders who placed bids on the awarded item who are ultimatelyawarded any item in the auction and generates a list thereof. That is,if bidder i is awarded a particular item in the auction, a list of allbidders who bid on the item awarded to bidder i is generated. (Note thatthe value for all the bidders who are not awarded an item in the auctionwill be zero.)

Subsequently, as shown at block 512, the identity of the winning bidderis removed from the list (i.e., the identity of bidder i is removed fromthe list). Next, as shown at block 514, the system determines the sum ofthe values of the bids for all other bidders in the auction who bid onthe item awarded to the winning bidder, that is, the sum of the valuesof the bids for the awarded items for all bidders whose identity remainson the list. This value is designated Ri.

Subsequently, the scenario is again evaluated, this time from aperspective as if the winning bidder had not participated in the auctionfor the awarded item, i.e., as if bidder i had not placed a bid on theitem it was ultimately awarded. This is shown at block 516. Next, thelist generated at block 510 is recalled by the system and the identityof the winning bidder (i.e., bidder i) and the second place bidder(i.e., the bidder who would have been the winning bidder had bidder inot participated in the auction) are removed from the list. This isshown at block 518. Subsequently, the sum of the values of the bids forall other bidders in the auction who bid on the item awarded to thewinning bidder, that is, the sum of the values of the bids for theawarded item for all bidders whose identity remains on the list, isdetermined, as shown at block 520. This value is designated Si.

For instance, in the above-described scenario, if content provider A hadnot participated in the auction, content provider B would have beenawarded pos₁ and thus, the sum of all other bidders who are awarded anyitem in the auction (in this case only content provider C) may bedetermined according to the altered allocation, the sum being denoted byS_(i).

Subsequently, the price the winning bidder must pay for the awarded itemis determined as Ri minus Si, as indicated at block 522. Accordingly, inthe above-described scenario, content provider A will be charged a pricefor pos₁ equal to the value taken away from content providers B and C.R_(i) in this scenario may be determined to be equal to $0.90 (the valueof content provider B's bid) plus $0.80 (the value of content providerC's bid), or $1.70. Additionally, if content provider A was removed fromthe determination and content provider B was awarded pos₁, Si (or thesum of the bid values for all other bidders who placed a bid on pos₁ andwho were ultimately awarded any item in the auction, in this case onlycontent provider C) may be determined to be equal to $0.80. Thus,content provider A will be charged R_(i) ($1.70) less S_(i) ($0.80) or$0.90 for placement in pos₁.

Note that in this scenario, the price that a bidder will be charged forbeing awarded an item generally will be the value of the second highestbid received for the item. In this instance, content provider A pays forpos₁ the bid value of content provider B's bid (the second highest bidvalue received for pos₁) or $0.90.

In certain circumstances, for instance when a large number of biddersare placing bids for a large number of items in a single auction, thegeneral Vickrey-like auction as described above can be somewhatdifficult to solve in real time. This is, in part, due to the need toseparately determine the allocation of items and the price for eachbidder who is awarded an item in the auction. In such circumstances, itmay be desirable to run a “modified” Vickrey-like auction. For amodified Vickrey-like auction, contemplate that the items beingauctioned may be provided in an order of decreasing desirability. Forinstance, in the above-described scenario, contemplate that the orderingof pos₁, pos₂, . . . pos_(k) is given in decreasing order ofdesirability. That is, if a content provider bids on position i and onposition j, then i<j implies that the content provider values position iover position j. As such, relative placement may be determined such thatfor each position in the order, a single-item Vickrey-like auction isperformed, and the winner simply removed from the equation forconsideration of subsequently determined positions.

In the modified Vickrey-like auction described herein, each bidder ispermitted to place only a single bid and designate such bid for one ormore items available for auction. A bid value may then be determined foreach bid received and the auction items may be allocated to the biddersin such a way that the total value is maximized. Accordingly, the valueof each bidder's bid determines, e.g., which position on the renderedpage it will be awarded. Thus, e.g., the content provider providing thebid with the highest bid value will be awarded the most prominentplacement (pos₁ in the above-described scenario), the content providerproviding the bid with the second highest bid value will be awarded thesecond most prominent placement (pos₂ in the above-described scenario),and so on. Note that the bid values to be awarded each contentprovider's bid may be determined, e.g., as previously set forthaccording to one or more of the cost-per-impression, cost-per-click,cost-per-sale, and/or revenue sharing pricing models.

By way of example only, contemplate that content provider A places a bidhaving a value of $1.00 and designates such bid for any of pos₁, pos₂,or pos₃; content provider B places a bid having a value of $0.90 anddesignates such bid for any of pos₁, pos₂, or pos₃; content provider Cplaces a bid having a value of $0.80 and designates such bid for any ofpos₁ or pos₂; content provider D places a bid having a value of $0.75and designates such bid for any of pos₁, pos₂, or pos₃; and contentprovider E places a bid having a value of $0.60 and designates such bidfor any of pos₁, pos₂, or pos₃.

With respect to pos₁, content provider A will be awarded the position ascontent provider A placed the bid having the highest bid value for pos₁.Content provider A will pay for pos₁ the value of the second highest bidfor pos₁ or $0.90 (content provider B's bid for pos₁).

Subsequently, with respect to pos₂, content provider A again providedthe bid having the highest bid value as its $1.00 bid was designated forany of pos₁, pos₂, or pos₃. However, in the modified Vickrey-likeauction of the present invention, it will be assumed that contentprovider A values pos₁ over pos₂ (as 1<2) and, thus, as content providerA has already been awarded pos₁, it will be removed from considerationfor subsequently determined positions. If content provider A is removedfrom consideration, of the remaining content providers, B, C, D and Eall placed bids for pos₂. Content provider B will be awarded pos₂ as itprovided the bid having the highest bid value (excluding contentprovider A) and content provider B will pay for pos₂, the second highestbid value placed for pos₂ (excluding content provider A) or $0.80 (thevalue of content provider C's bid for pos₂).

Subsequently, with respect to pos₃, content provider A again providedthe bid having the highest bid value as its $1.00 bid was designated forany of pos₁, pos₂, or pos₃. However, as content provider A has alreadybeen awarded a position, it will be removed from the consideration fordetermination of the content item placed in pos₃. Of the remainingcontent providers, content provider B provided the bid having thehighest bid value for pos₂ as its $0.90 bid was designated for any ofpos₁, pos₂, or pos₃. However, content provider B has also already beenawarded a position (pos₂), and, therefore, it will be removed fromconsideration for determination of the content item placed in pos₃ aswell.

Of the remaining content providers, only content providers D and Eplaced bids for pos₃ (content provider C's bid being designated only forpos₁ or pos₂). As such, content provider D will be awarded pos₃ (as itsbid for pos₃ is the highest of the remaining content providers beingconsidered for pos₃) and it will pay for pos₃ the second highest bidvalue placed for pos₃ (excluding content providers A and B) or $0.60(the value of content provider E's bid for pos₃).

In yet another embodiment of the present invention, a content providermay be permitted to specify a different bid for each item on which itdesires to place a bid and the allocation and pricing of the itemsdetermined according to the modified Vickrey-like auction principleshereinabove described. For instance, again contemplate theabove-described scenario wherein content providers A, B, C, D, and E arebidding for placement of their content in pos₁, pos₂, and pos₃ on arendered page. Further contemplate that content provider A places a bidhaving a value of $1.00 for pos₁ and a value of $0.49 for pos₂, contentprovider B places a bid having a value of $0.90 for pos₁ and a bidhaving a value of $0.50 for pos₂, and content provider C places a bidhaving a value of $0.75 for pos₁ and a bid having a value of $0.30 forpos₂, the remaining content providers and bids for pos₃ being irrelevantto the present example and, accordingly, not delineated herein.

With respect to pos₁, content provider A will be awarded the position asit placed the bid having the highest bid value for pos₁ ($1.00) and willpay for pos₁ the value of the second-highest bid for the position or$0.90 (content provider B's bid for pos₁). With respect to pos₂, contentprovider B will be awarded the position as it placed the bid having thehighest bid value for pos₂ ($0.50). However, in determining the pricethat content provider B must pay for pos₂, content provider A's bid of$0.49 for pos₂ will be taken out of consideration for determination ofthe content item placed in pos₂. This is because under the modifiedVickrey-like auction of this embodiment of the present invention, it isassumed that content provider A values pos₁ over pos₂ (as 1<2) and, ascontent provider A was already awarded pos₁, it will be taken out of thedetermination for pos₂. Thus, content provider B will pay for pos₂ thesecond highest bid value for pos₂ (excluding content provider A) or$0.30 (the value of content provider C's bid for pos₂).

Methods for Introducing Randomness into the Determination of theRelative Placement of Content Items on a Rendered Page

In another embodiment, the present invention relates to systems andmethods for introducing randomness into the relative placement ofcontent items based upon bid value, the content items to be placed on apage (e.g., a web page) rendered according to a given context, forinstance, the results of a particular query run by a search engine. Inthe relative placement scenarios described hereinabove, content itemsare ranked and selected for relative placement on a rendered page basedsolely upon the bid values determined for each content item. That is,the bid values determined for each content item are ranked in descendingorder and, if there are N positions on a rendered page available forplacement of content items, the N content items having the highest bidvalues are selected and placed in the N positions on the page. Thecontent item that has the highest bid value is placed in the mostprominent or desired position, the content item that has the secondhighest bid value is placed in the second most prominent or desiredposition, and so on. The methodologies on which these scenarios are runare deterministic and will render the same result each and every timethey are run from the same information.

In one embodiment of the present invention, randomness is introducedinto the deterministic scenarios hereinabove described by permittingeach content provider placing a bid on a particular placement positionon the rendered page, to have its content item allocated to thatparticular placement position at a frequency proportional to theexpected performance of its content item relative to the expectedperformance of the content items of all content providers placing bidsfor the same position. With reference to FIG. 6, a flow diagram isillustrated which shows a computer-implemented method 600 forintroducing randomness into the relative placement allocation of contentitems in accordance with this embodiment of the present invention.

Initially, as shown at block 610, the system receives a plurality ofbids for a particular position on a rendered page, each bid beingassociated with a content item. Subsequently, a bid value is determinedfor each of the plurality of bids, as indicated at block 612.

For instance, contemplate the above-described scenario wherein aplurality of content providers are bidding for placement of theircontent items on a rendered page relative to the content items of othercontent providers and wherein the positions available to the contentproviders may be ordered as pos₁, pos₂, . . . , pos_(k), with pos₁ beingthe top-most position on the rendered page, pos₂ being the secondposition from the top of the rendered page, and so on. Furthercontemplate that four content providers, A, B, C, and D place bids forplacement of their content item to be placed in pos₁, the bid value forcontent provider A's content item being equal to $5.00, the bid valuefor content provider B's content item being equal to $4.00, the bidvalue for content provider C's content item being equal to $3.00 and thebid value for content provider D's content item being equal to $2.00.

Subsequently, as indicated at block 614, the expected performance ofeach content item based upon the bid value associated therewith isdetermined relative to the bid values of the bids associated with allother content items for which bids were placed for placement in pos₁.That is, the relative expected performance of each content item,REP_(x), is determined according to the following formula:${REP}_{X} = \frac{\left( {{Bid}\quad{Value}\quad{of}\quad{Content}{\quad\quad}{Item}\quad X} \right)^{\alpha}}{\sum\left( {{Bid}\quad{Value}\quad{of}\quad{All}\quad{Content}\quad{Items}\quad{Being}\quad{Ranked}} \right)^{\alpha}}$

the significance of the value of the parameter being more fullydescribed below.

Thus, returning to the above-described scenario, the relative expectedperformance of content item A, REP_(A), (with =1) may be determined as:${REP}_{A} = {\frac{\left( {{\$ 5}{.00}} \right)^{1}}{\left( {{{\$ 5}{.00}} + {{\$ 4}{.00}} + {{\$ 3}{.00}} + {{\$ 2}{.00}}} \right)^{1}} = 0.3571}$

The expected performance of content item A can therefore claim 35.71% ofthe total value among all content items bidding for placement in pos₁.Performing a similar calculation for each of content items B, C, and D,the relative expected performance of each may be determined such thatcontent item B can claim 28.57% of the total value among all contentitems bidding for placement in pos₁, content item C can claim 21.43% ofthe total value, and content item D can claim 14.29% of the total value.

Subsequently, as shown at block 616, the particular position for whicheach content provider's bid was placed (i.e., pos₁) may be allocated toeach of content items A, B, C, and D in proportion to their relativeexpected performance. That is, content item A will be awarded pos₁35.71% of the time, content item B will be awarded pos₁ 28.57% of thetime, content item C will be awarded pos₁ 21.43% of the time and contentitem D will be awarded pos₁ 14.29% of the time, the awarding of pos₁occurring proportionately in terms of frequency but completely atrandom.

One way in which the above-described allocation methodology may bethought of is to contemplate that lottery tickets are awarded to each ofthe bidder's items (e.g., content items) for which bids are placed onthe particular auctioned item (e.g., pos₁), the number of lotterytickets awarded being based determined in proportion to theirperformance relative to other bidders placing bids for the sameauctioned item (e.g., other content providers placing bids for placementin the same position on the rendered page), and a simulated randomlottery drawing is performed to choose which bidder's item (e.g.,content item) is awarded the auctioned item (e.g., pos₁). In theabove-described scenario, the number of lottery tickets awarded and therelative performance of all content items in a lottery having 1,000tickets may be determined as set forth in Table I. TABLE I ContentExpected Number of Provider/ Bid Relative Lottery Item Value PerformanceTickets A $5.00 35.71% 357.1 B $4.00 28.57% 285.7 C $3.00 21.43% 217.3 D$2.00 14.29% 142.9

In this example, each content item has been allocated a number oftickets proportional to their value relative to the whole. When awinning ticket is chosen randomly, content items A will have a 35.71%chance of being selected, content item B will have a 28.57% chance ofbeing selected, content item C will have a 21.73% chance of beingselected, and content item D will have a 14.29% chance of beingselected. Subsequently, as shown at block 618, each of the content itemsmay be displayed on the rendered page in proportion to its relativeexpected performance, the awarding of pos₁ occurring proportionately interms of frequency but completely at random.

The parameter permits control over how much “exploring” vs. how much“exploiting” is done. That is, as the parameter increases in value, thelottery allocation becomes more skewed toward higher performing bidders(e.g., the content items of higher performing content provider's),making the lottery more like a standard rank with little or norandomization (i.e., “exploiting”). Conversely, lowering the value ofthe parameter causes increased randomness, permitting lower-performingcontent items to be selected for display more frequently, thus givingthem a chance to improve their standing (i.e., “exploring”).

For instance, if an parameter of three is applied to the relativeexpected performance equation in the above-described example whereinthere are four content items A, B, C, and D having bid values of $5.00,$4.00, $3.00, and $2.00, respectively, the number of lottery ticketsallotted to each of the content items would be as shown in Table II.Notice how content item A, the highest performer, is awarded a largernumber of tickets than when the parameter had a value of one. Thus, thehigher value for the parameter exploits the higher performers and,consequently, penalizes the lower performing content provider's items.TABLE II Content Expected Number of Provider/ Bid Relative Lottery ItemValue Performance Tickets A $5.00 55.80% 558.0 B $4.00 28.57% 285.7 C$3.00 12.05% 120.5 D $2.00  3.57%  35.7

Introducing randomness into content ranking may seem counterintuitive.However, it makes sense in a scenario wherein the providers of contentitems (e.g., advertisements or images) are bidding for positions on arendered page (e.g., a web page) for a number of reasons. First, in astandard ranking determination, if N placement positions are availableon the rendered page, only the top N results will be displayed to theuser. Content items not in the top N are either displayed on asubsequent search results page, or not displayed at all. This poses aproblem when new content items enter the system since ranking is basedon performance and there is no recorded performance for a new contentprovider. One way in which this problem may be countered when using astandard ranking determination is to give an artificial boost to newcontent providers or items, thereby ensuring that some performance isrecorded. This solution is sub-optimal to the method of the presentinvention, however, since new content items are treated differently thanlong-running content items and content providers may be able to takeadvantage of this, perhaps unfairly.

A second reason that it makes sense to introduce randomness into ascenario wherein the providers of content items are bidding forpositions on a rendered page is that one danger of standard rankingdetermination for search advertising is that smaller content providers(i.e., advertisers) may be pushed out by content providers who havelarger marketing budgets or are willing to take loss-leading bids topush out competition. The method of the present invention whereinrandomness is introduced into the determination avoids this problem byproviding all content providers at least some inventory in a fairmanner, that is, in proportion to their bid value. This should increasecompetition and stop bid prices from bottoming out.

Methods for Incremental Pricing of Items in an Auction Based UponDelineated Attributes

In another embodiment, the present invention relates to systems andmethods for determining bid values based upon one or more targetingattributes for which a bidder (e.g., a content provider) may desire toplace an incrementally different bid. Auctions often include a largenumber of basic goods that are similar but not exact. That is, the basicgoods may contain a high number of variable attributes which make themincrementally different from other similar goods. In such situations, abidder may wish to place a higher or lower bid on goods having one ormore particular attributes than on similar goods that are void of suchattributes. Accordingly, the method of this embodiment of the presentinvention permits a bidder to allocate a “base bid” for each basic goodand then incrementally adjust the base bid for each attribute that itperceives as being of increased or decreased value. At the time theauction is run for each good, the total for each bidder's base bid andall of the incremental values which hold true regarding the good'sattributes may be summed and the bid value determined.

This embodiment of the present invention may be particularly useful forbusiness-to-business marketplaces in which thousands of similar but notexact goods change hands on a regular basis. In a currently preferredembodiment, targeting criteria with respect to search engine usersplacing queries for particular content items and/or the relativeplacement thereof may be differentially priced separately from the basebids for the search terms themselves. However, it will be understood andappreciated by those of ordinary skill in the art that the method ofthis embodiment of the present invention also may be used for othergoods in which there may be a basic rate and add-on options, e.g.,bicycles, machine parts, or automobiles.

With reference to FIG. 7, a flow diagram is illustrated which shows acomputer-implemented method 700 for determining bid values based uponone or more targeting attributes in accordance with this embodiment ofthe present invention. It will be understood and appreciated by those ofordinary skill in the art that although the FIGS. and discussion of thisembodiment of the present invention refer to the above-describedscenario wherein a number of content providers are bidding for placementof one or more content items (e.g., advertisements or images) on arendered page (e.g., a web page), the methods and systems hereindescribed are equally applicable in other auction settings. Theparticular scenario described herein is exemplary only and is notintended to limit the present invention in any way.

Initially, as shown at block 710, the system receives a base bid forplacement of a bidder's content item on a rendered page. Subsequently,it is determined whether or not incremental pricing is desired basedupon targeted attributes, as shown at block 712. If incremental pricingis not desired, a bid value for the bid is determined based solely onthe base bid. This is indicated at block 714. If incremental pricing isdesired, however, the attributes for which pricing increments aredesired, as well as the value of such pricing increments, must bedetermined, as more fully described below. It should be noted that useherein of the terms “increment” or “incremental” is intended to covernot only increments in pricing based upon particular attributes and/orplacement positions but also pricing decrements as well for, e.g.,undesirable attributes and/or positioning.

Referring to FIG. 8A, an illustrative screen display 800 of an exemplaryuser interface (UI) is shown on which such base bids (and theirassociated search phrases or bid terms) may be entered or selected by abidder and displayed. It will be understood and appreciated by those ofordinary skill in the art that this is only one possible visualconfiguration and, accordingly, is not intended to limit the scope ofthe invention in any way. A number of conceivable configurations may bepresented which permit the bidder (i.e., the content provider) to enterthe desired information and all such variations are contemplated to bewithin the scope of the present invention.

Screen display 800 includes a base bid entry/display area 810 and atargeting attribute selection area 812. The base bid entry/display area810 includes a desired search phrase entry/display area 814, a desiredbase bid entry/display area 816, and a desired match typeindication/display area 818. The desired search phrase entry/displayarea 814 permits a bidder (e.g., a content provider) to enter the searchphrase or bid term for which placement of a bid is desired. In thescreen display 800 of FIG. 8A, the bidder has entered the search phrase“Steel Belted Radial” in the desired search phrase entry/display area814. The desired base bid entry/display area 816 permits the bidder toenter the base bid that they wish to place for the search term or phaseentered. In the screen display 800 of FIG. 8A, the bidder has entered abase bid of $0.25 in the desired base bid entry/display area 816. Thedesired match type indication/display area 818 permits a bidder toindicate in which type of match the entered search term or phrase isdesired to be included. A “broad” match is indicated in the desiredmatch type indication/display area 818 of the screen display 800 of FIG.8A.

The targeting attribute selection area 812 of the screen display 800 ofFIG. 8A includes a plurality of targeting attributes for which the basebid may be incremented or decremented. For instance, the targetingattribute selection area 812 of the screen display 800 of FIG. 8Aincludes a user location targeting area 820, a day of the week targetingarea 822, a time of day targeting area 824, and a relative placementposition targeting area 826. It will be understood and appreciated bythose of ordinary skill in the art that the targeting attributes shownin the screen display 800 of FIG. 8A are merely illustrative and are notintended to limit the scope of the present invention in any way.

Once the search phrase or bid term, the match type, and the base bidhave been entered or selected by the bidder, if it is determined atblock 712 (FIG. 7) that incremental pricing based upon targetedattributes is not desired, the bidder may simply select the “Next”indicator 828 on the exemplary screen display 800 of FIG. 8A and proceedwith the determination of bid value based solely upon the base bid. Aspreviously discussed, this is shown at block 714 of FIG. 7. It should benoted that if the bidder determines that they do not wish to proceedwith placing a bid for the search phrase or bid term, they may exit thebidding system by selection of the “Cancel” indicator 830 on theexemplary screen display 800 of FIG. 8A.

Returning to FIG. 7, if it is determined at block 712 that incrementalpricing based upon targeted attributes is desired, the system receivesan indication for which of a plurality of user targeting attributesincremental pricing is desired. This is shown at block 716. Once adesired targeting attribute has been received, one or more attributeoptions for the desired targeting attribute may be displayed to thebidder, as indicated at block 718. The system may then receive anindication of the desired attribute options for which the bidder desiresincremental pricing, as indicated at block 720, as well as an indicationof the desired pricing increment (or decrement) for each desiredattribute option, as indicated at block 722. Once all targetingattributes and their associated attribute options for which the bidderdesires bid pricing adjustments have been received by the system, thebid value for the targeted bid may be determined. This is indicated atblock 724.

Returning to the exemplary screen display 800 of FIG. 8A, if the bidderdesires incremental pricing based upon targeted attributes, it mayselect one or more of the user location targeting area 820, the day ofthe week targeting area 822, the time of day targeting area 824 and therelative placement position targeting area 826. If the bidder selectsthe user location targeting area 820, the bidder may then be presentedwith an additional exemplary screen display 800 a, as shown in FIG. 8B.The exemplary screen display 800 a of FIG. 8B includes a selectedattribute display area 832, an attribute option display area 834, and abase bid display and instruction area 836. The selected attributedisplay area 832 indicates that the bidder has selected the userlocation targeting area 820 of the screen display 800 of FIG. 8A and,thus, the bidding system is providing targeted bidding options basedupon user location. The base bid display and instruction area 836indicates that the bidder indicated a base bid of $0.25 in the base bidentry/display area 816 of the screen display 800 of FIG. 8A and informsthe bidder that the increments selected in the attribute option displayarea 834 will be applied in addition to the base bid when the indicatedcriteria are met, as more fully described below.

The attribute option display area 834 of the screen display 800 a ofFIG. 8B includes a plurality of attribute option slots 838 the biddermay use to increment their base bid. The down arrow 840 to the right ofeach of the attribute option slots 838 shown indicates that the biddermay select from a pre-defined list of options in each of the attributeoption slots 838. However, it will be understood by those of ordinaryskill in the art that the bidder may be permitted to enter their ownoptions rather than selecting from a pre-defined list, if desired. Theattribute option display area 834 further includes a pricing incremententry/display area 842 associated with each of the attribute optionslots 838 wherein the bidder may indicate the amount of any increment(or decrement) they wish to associate with the attribute optiondisplayed. For instance, in the top attribute option slot 838 of theattribute option display area 834 of the screen display 800 a of FIG.8B, the bidder has indicated that the bid will be incremented by $0.10for each search engine user that has an Internet Protocol (IP) addressin New York, N.Y.

Once the bidder has entered/selected all desired attribute options andtheir associated bid pricing increments within the user locationtargeting area 820 (FIG. 8A), the “Submit” indicator 844 may beselected, at which time the bidder will be returned to the exemplaryscreen display 800 of FIG. 8A. Note that at any time if the bidderdetermines that they do not wish to increment their bid based upon userlocation, the “Cancel” indicator 846 may be selected to return thebidder to the exemplary screen display 800 of FIG. 8A.

Once returned to the exemplary screen display 800 of FIG. 8A, the biddermay select another of the targeted attributes on which bid adjustmentsare desired, i.e., the day of the week targeting area 822, the time ofday targeting area 824, and/or the relative placement position targetingarea 826 may be selected (or the bidder may return to the user locationtargeting area 820, if desired). If the bidder selects the day of theweek targeting area 822, the bidder may then be presented with anadditional exemplary screen display 800 b, as shown in FIG. 8C. Theexemplary screen display 800 b of FIG. 8C includes a selected attributedisplay area 832 a, an attribute option display area 834 a, and a basebid display and instruction area 836 a. The selected attribute displayarea 832 a indicates that the bidder has selected the day of the weektargeting area 822 of the screen display 800 of FIG. 8A and, thus, thebidding system is providing targeted bidding options based upon the dayof the week a user is placing a search query. The base bid display andinstruction area 836 a indicates that the bidder indicated a base bid of$0.25 in the base bid entry/display area 816 of the screen display 800of FIGS. 8A and informs the bidder that the increments selected in theattribute option display area 834 a will be applied in addition to thebase bid the indicated criteria are met, as more fully described below.

The attribute option display area 834 a of the exemplary screen display800 b of FIG. 8C includes a plurality of attribute option slots 838 athe bidder may use to increment their base bid. As with the attributeoption display area 834 of the exemplary screen display 800 a of FIG.8B, a down arrow 840 a to the right of each of the attribute optionslots 838 a indicates that the bidder may select from a pre-defined listof options in each of the attribute option slots 838 a. However, it willbe understood and appreciated by those of ordinary skill in the art thatthe bidder may be permitted to enter their own options rather thanselecting from a pre-defined list, if desired. The attribute optiondisplay area 834 a further includes a pricing increment entry/displayarea 842 a associated with each of the attribute option slots 838 awherein the bidder may indicate the amount of any increment (ordecrement) they wish to associate with the attribute option displayed.For instance, in the top attribute option slot 838 a of the attributeoption display area 834 a of the screen display 800 b of FIG. 8C, thebidder has indicated that the bid will incremented by $0.10 for eachsearch engine user that is placing their search engine query on aMonday.

Once the bidder has entered/selected all desired attribute options andtheir associated bid pricing increments within the day of the weektargeting area 822 (FIG. 8A), the “Submit” indicator 844 a may beselected at which time the bidder will be returned to the exemplaryscreen display 800 of FIG. 8A. Note that at any time if the bidderdetermines that they do not wish to increment their bid based upon theday of the week, the “Cancel” indicator 846 a may be selected to returnthe bidder the exemplary screen display 800 of FIG. 8A.

Once returned to the exemplary screen display 800 of FIG. 8A, the biddermay select another of the targeted attributes on which bid adjustmentsare desired, i.e., the time of day targeting area 824 and/or therelative placement position targeting area 826 may be selected (or thebidder may return to the user location targeting area 820 or the day ofthe week targeting area 822, if desired). If the bidder selects the timeof day targeting area 824, the bidder may then be presented with anadditional exemplary screen display 800 c, as shown in FIG. 8D. Theexemplary screen display 800 c of FIG. 8D includes a selected attributedisplay area 832 b, an attribute option display area 834 b, and a basebid display and instruction area 836 b. The selected attribute displayarea 832 b indicates that the bidder has selected the time of daytargeting area 824 of the screen display 800 of FIG. 8A and, thus, thebidding system is providing targeted bidding options based upon the timeof day a user is placing a search query. The base bid display andinstruction area 836 b indicates that the bidder indicated a base bid of$0.25 in the base bid entry/display area 816 of the screen display 800of FIG. 8A and informs the bidder that the increments selected in theattribute option display area 834 b will be applied in addition to thebase bid when the indicated criteria are met, as more fully describedbelow.

The attribute option display area 834 b of the exemplary screen display800 c of FIG. 8D includes a plurality of attribute option slots 838 bthe bidder may use to increment their base bid. As with the attributeoption display area 834 of the exemplary screen display 800 a of FIG.8B, a down arrow 840 b to the right of each of the attribute optionslots 838 b indicates that the bidder may select from a pre-defined listof options in each of the attribute option slots 838 b. However, it willbe understood and appreciated by those of ordinary skill in the art thatthe bidder may be permitted to enter their own options rather thanselecting from a pre-defined list, if desired. The attribute optiondisplay area 834 b further includes a pricing increment entry/displayarea 842 b associated with each of the attribute option slots 838 bwherein the bidder may indicate the amount of any increment (ordecrement) they wish to associate with the attribute option displayed.For instance, in the top attribute option slot 838 b of the attributeoption display area 834 b of the screen display 800 c of FIG. 8D, thebidder has indicated that bid will be incremented by $0.10 for eachsearch engine user that is placing their search engine query at or about12:00 p.m.

Once the bidder has entered/selected all attribute options and theirassociated bid pricing increments that are desired within the time ofday targeting area 824 (FIG. 8A), it may select the “Submit” indicator844 b may be selected at which time the bidder will be returned to theexemplary screen display 800 of FIG. 8A. Note that at any time if thebidder determines that they do not wish to increment their bid basedupon the time of day, the “Cancel” indicator 846 b may be selected toreturn the bidder to the exemplary screen display 800 of FIG. 8A.

Once returned to the exemplary screen display 800 of FIG. 8A, the biddermay select another of the targeted attributes on which bid adjustmentsare desired, i.e., the relative placement position targeting area 826may be selected (or the bidder may return to the user location targetingarea 820, the day of the week targeting area 822, or the time of daytargeting area 824, if desired). If the bidder selects the relativeplacement position targeting area 826, the bidder may then be presentedwith an additional exemplary screen display 800 d, as shown in FIG. 8E.The exemplary screen display 800 d of FIG. 8E includes a selectedattribute display area 832 c, an attribute option display area 834 c,and a base bid display and instruction area 836 c. The selectedattribute display area 832 c indicates that the bidder has selected therelative placement position targeting area 826 of the screen display 800of FIG. 8A and, thus, the bidding system is providing targeted biddingoptions based upon the relative placement position that may be awardedthe content item associated with the bid placed. The base bid displayand instruction area 836 c indicates that the bidder indicated a basebid of $0.25 in the base bid entry/display area 816 of the screendisplay 800 of FIG. 8A and informs the bidder that the incrementsselected in the attribute option display area 834 c will be applied inaddition to the base bid when the bid placed meets the indicatedcriteria, as more fully described below.

The attribute option display area 834 c of the exemplary screen display800 d of FIG. 8E includes a plurality of attribute option slots 838 cthe bidder may use to increment their base bid. As with the attributeoption display area 834 of the exemplary screen display 800 a of FIG.8B, a down arrow 840 c to the right of each of the attribute optionslots 838 c indicates that the bidder may select from a pre-defined listof options in each of the attribute option slots 838 c. However, it willbe understood and appreciated by those of ordinary skill in the art thatthe bidder may be permitted to enter their own options rather thanselecting from a pre-defined list, if desired. The attribute optiondisplay area 834 c further includes a pricing increment entry/displayarea 842 c associated with each of the attribute option slots 838 cwherein the bidder may indicate the amount of any increment (ordecrement) they wish to associate with the attribute option displayed.For instance, in the top attribute option slot 838 c of the attributeoption display area 834 c of the screen display 800 d of FIG. 8E, thebidder has indicated that their bid will be incremented by $0.85 forplacement in the prominent or most desired content item placementposition.

Once the bidder has entered/selected all desired attribute options andtheir associated bid pricing increments within the relative placementposition targeting area 826 (FIG. 8A), the “Submit” indicator 844 c maybe selected at which time the bidder will be returned to the exemplaryscreen display 800 of FIG. 8A. Note that at any time if the bidderdetermines that they do not wish to increment their bid based uponrelative placement position of the content item, the “Cancel” indicatormay be selected 846 c to return the bidder to the exemplary screendisplay 800 of FIG. 8A.

In the exemplary screen displays 800 a, 800 b, 800 c, and 800 d, anyintersection of one or more attribute options results in addition (orsubtraction) of the indicated increments. For instance, if a search forthe phrase “steel belted radial” ($0.25) is submitted from an InternetProtocol (IP) address in New York, N.Y. ($0.10) on Monday ($0.10) at orabout 12:00 noon ($0.10) and the content item is awarded the mostprominent placement position as a result of the search ($0.85), thetotal bid would be $1.40. From this bid, the bid value may bedetermined, depending upon the pricing model under which it wassubmitted (e.g., CPI bid, CPC bid, CPS bid, and/or revenue sharingpercentage bid).

Using the method of this embodiment of the present invention, contentproviders may be provided with the flexibility to bid on a basic good(i.e., a search term) and then incrementally adjust their bid based upondesired attributes, e.g., the search engine user and/or the awardedplacement position. The actual bid values may then be determineddynamically depending on which attribute options are satisfied.

Method for Determining Relative Placement of Content Items of VaryingShapes and Sizes

In another embodiment, the present invention relates to systems andmethods for determining relative placement of content items of varyingshapes and sizes based upon bid value, the content items to be placed ona page (e.g., a web page) rendered according to a given context, forinstance, the results of a particular query run by a search engine. Inthe auction scenarios hereinabove described, content providers arepermitted to place bids for one or more similarly defined, althoughdisparately located, content item positions on a page. However, suchsize and shape limitations are often not conducive to the content item aparticular content provider wishes to place on the rendered page. Forinstance, a content provider may have a content item that is twice astall as a standard content item and, accordingly, it would value twoadjacent slots (one linearly atop the other), but award of a singlecontent item position would be useless.

In one embodiment of the present invention, content providers may bepermitted to bid for mixed or bundled content item positions if, e.g.,the content item desired to be placed on the rendered page does not fitwithin the parameters of a single defined content item position.Determination of which content item positions are awarded which contentitems (or, more accurately, which content providers) may still bedetermined such that value to the hosting site (e.g., the search engine)is maximized, as more fully described below.

Referring to FIG. 9, a flow diagram is illustrated which shows acomputer-implemented method 900 for determining relative placement ofcontent items of varying shapes and sizes in accordance with anembodiment of the present invention. Initially, as indicated at block910, the system receives a plurality of bids, each bid being associatedwith a content item. Subsequently, as indicated at block 912, the systemreceives an indication of which one or more of a plurality of contentitem positions each of the plurality of bids is to be associated. Eachbid received may be associated with the same content item position orcombination of content item positions or the bids received may vary withregard to which content item position or positions they are to beassociated.

Next, as shown at block 914, a bid value may be determined for each ofthe plurality of bids, if necessary. It will be understood by those ofordinary skill in the art that if all bids received were placedaccording to the same pricing model (e.g., the cost-per-impressionpricing model, the cost-per-click pricing model, the cost-per-salepricing model, or the revenue sharing pricing model), the bids could beconsidered to represent comparable value and determination of a bidvalue therefore may not be necessary.

It should be noted with respect to bid value determination that othercriteria, for instance, an increased likelihood that a larger contentitem will be accessed relative to a smaller content item, may be takeninto account as well, if desired.

Subsequently, as shown at block 916, the plurality of content itempositions may be allocated among the content items for which bids werereceived such that value to the hosting site (e.g., the search engineweb site) is maximized, as more fully described below. Next, asindicated at block 918, the content items may be displayed in theirrespective allocated content item positions.

Referring to FIG. 10, a schematic diagram is shown which illustrates anexample in which the bundled auction method of this embodiment of thepresent invention may be applied to five content providers (A, B, C, D,and E) each of which place bids for placement of their content items ona single rendered page (e.g., a web page). In this example, each of thecontent providers desire to have content items of varying sizes andshapes displayed on the rendered page and, accordingly, desire to bid ondifferent combinations of available space on the page.

FIG. 10 illustrates a web-page layout 1000 in the form of grid havingfour content item positions (A1, A2, B1, and B2), each of which isrectangular in shape and approximately equal in size, arranged adjacentone another such that a larger overall rectangle is formed. It will beunderstood and appreciated by those of ordinary skill in the art thatthe web-page layout 1000 illustrated in FIG. 10 is merely exemplary andnot intended to limit the scope of the present invention in any way. Forinstance, the matrix layout may be arbitrarily large. Additionally,arbitrary geometries may also be rendered (e.g., on an L, T, or gammageometry). All such variations are contemplated to be within the scopeof the present invention.

By way of example, contemplate a scenario wherein content providers Aand B each wish to bid for placement of content items on the web-pagelayout 1000 and each have a content item which approximates the size andshape of any one of the four content item positions A1, A2, B1, and/orB2. Accordingly, content providers A and B place bids (whether identicalor varying based upon position) for placement of their content items ina single content item position. It will be understood and appreciated bythose of ordinary skill in the art that the bids received from each ofcontent providers A and B may be based on any one or more of theabove-described pricing models (i.e., the CPI pricing model, the CPCpricing model, the CPS pricing model, and/or the revenue share pricingmodel) and a bid value determined therefore, if desired.

For purposes of the present example, contemplate that content provider Avalues each of the available content item positions equally and places abid for any of content item positions A1, A2, B1, or B2 having a bidvalue of $3.00. Further contemplate that content provider B valuescontent item positions A1 or B1 over content item positions A2 or B2.Accordingly, content provider B places a bid for either of positions A1or B1 having a bid value of $4.00 and for either of A2 or B2 having abid value of $2.00.

Further contemplate that content provider C wishes to bid for placementof a content item on the web-page layout 1000 and has a content itemwhich approximates the size and shape of two of the content itempositions positioned linearly atop one another. That is, contentprovider C desires to place a bid for placement of its content item suchthat it encompasses either the combination of positions A1 and A2 or thecombination of content positions B1 and B2. Accordingly, contentprovider C places bids (whether identical or varying based upon which ofthe combinations it is awarded) for placement of its content item on theweb-page layout 1000 of FIG. 10. It will again be understood by those ofordinary skill in the art that the bids received from content provider Cmay be based on any one or more of the above-described pricing modelsand a bid value determined therefore, if desired.

For purposes of the present example, contemplate that content provider Cvalues each of the combination of content items A1 and A2 and thecombination of content B1 and B2 equally and places a bid for eithercombination having a bid value of $7.00.

Further contemplate that content provider D wishes to bid for placementof a content item on the web-page layout 1000 and has a content itemwhich approximates the size and shape of two of the content itempositions positioned laterally aside one another. That is, contentprovider D desires to place a bid for placement of its content item suchthat it encompasses either the combination of positions A1 and B1 or thecombination of content positions A2 and B2. Accordingly, contentprovider D places bids (whether identical or varying based upon which ofthe combinations it is awarded) for placement of its content item on theweb-page layout 1000 of FIG. 10. It will again be understood andappreciated by those of ordinary skill in the art that the bids receivedfrom content provider D may be based on any one or more of theabove-described pricing models and a bid value determined therefore, ifdesired.

For purposes of the present example, contemplate that content provider Dvalues each of the combination of content items A1 and B1 and thecombination of content items A2 and B2 equally and places a bid foreither combination having a value of $6.00

Further contemplate that content provider E wishes to bid for placementof a content item on the web-page layout 1000 and has a content itemwhich approximates the size and shape of all four of the content itempositions arranged as they are on the web-page layout 1000. That is,content provider E desires to place a bid for placement of its contentitem such that it encompasses the combination of all of content itempositions A1, A2, B1, and B2 and, accordingly, places a bid therefore(such bid being based upon any one or more of the above-describedpricing models). For purposes of the present example, contemplate thatcontent provider E places a bid having a value of $10.00 for thecombination of all of content items A1, A2, B1, and B2.

As previously described, the allocation of content item positions to thevarious content providers will be determined such that value to thehosting site (e.g., the search engine web site) is maximized.Accordingly, the system of the present invention must determine which ofthe available combinations of space maximizes the total value andallocate the content item positions accordingly. In a first scenario,contemplate that content provider E is awarded the combination all ofcontent item positions A1, A2, B1, and B2. As all content item positionsare allocated to content provider E, content providers A, B, C, and Dwill not have their content items displayed in the web page layout 1000of FIG. 10. The total value in this first scenario is the value ofcontent provider E's bid or $10.00.

In a second scenario, contemplate that content provider D is awarded thecombination of content item positions A1 and B1 (which it values at$6.00). Content items A2 and B2 remain to be allocated. As contentprovider E bid only on the combination of all content items, theremaining content items to be allocated are of no interest to it.Additionally, as content provider C placed bids on only the combinationof content items A1 and A2 or B1 and B2, the remaining content items tobe allocated are of no interest to it either. Content providers A and Bplaced bids on any one of the content item positions individually.Assume for purposes of this example that each content provider may beawarded only a single content item or combination of items on which itplaced a bid. As such, value is maximized equally if content provider Ais awarded content item A2 and content provider B is awarded contentitem B2 or if content provider B is awarded content item A2 and contentprovider A is awarded content item B2. In either instance, the valuereceived for content items A2 and B2 is $5.00. Thus, the overall valueof this second scenario is $6.00 plus $5.00 or $11.00.

In a third scenario, contemplate that content provider D is awarded thecombination of content item positions A2 and B2 (which it values at$6.00). Content items A1 and B1 remain to be allocated. As contentprovider E bid only on the combination of all content items, theremaining content items to be allocated are of no interest to it.Additionally, as content provider C placed bids on only the combinationof content items A1 and A2 or B1 and B2, the remaining content items tobe allocated are of no interest to it either. Content providers A and Bplaced bids on any one of the content item positions individually. Sinceit is assumed for purposes of this example that each content providermay be awarded only a single content item or combination of items onwhich it placed a bid, value is maximized equally if content provider Ais awarded content item A1 and content provider B is awarded contentitem B1 or if content provider B is awarded content item A1 and contentprovider A is awarded content item B1. In either instance, the valuereceived for content items A1 and B1 is $7.00. Thus, the overall valueof this third scenario is $6.00 plus $7.00 or $13.00.

In a fourth scenario, contemplate that content provider C is awarded thecombination of content item positions A1 and A2 (which it values at$7.00). Content items B1 and B2 remain to be allocated. As contentprovider E bid only on the combination of all content items, theremaining content items to be allocated are of no interest to it.Additionally, as content provider D placed bids on only the combinationof content items A1 and B1 or A2 and B2, the remaining content items tobe allocated are of no interest to it either. Content providers A and Bplaced bids on any one of the content item positions individually. Sinceit is assumed for purposes of this example that each content providermay be awarded only a single content item or combination of items onwhich it placed a bid, value is maximized if content provider A isawarded content item B2 (which it values at $3.00) and content providerB is awarded content item B1 (which it values at $4.00). Thus, theoverall value of this fourth scenario is $7.00 plus $3.00 plus $4.00 or$14.00.

In a fifth scenario, contemplate that content provider C is awarded thecombination of content item positions B1 and B2 (which it values at$7.00). Content items A1 and A2 remain to be allocated. As contentprovider E bid only on the combination of all content items, theremaining content items to be allocated are of no interest to it.Additionally, as content provider D placed bids on only the combinationof content items A1 and B1 or A2 and B2, the remaining content items tobe allocated are of no interest to it either. Content providers A and Bplaced bids on any one of the content item positions individually. Sinceit is assumed for purposes of this example that each content providermay be awarded only a single content item or combination of items onwhich it placed a bid, value is maximized if content provider A isawarded content item A2 (which it values at $3.00) and content providerB is awarded content item A1 (which it values at $4.00). Thus, theoverall value of this fifth scenario is $7.00 plus $3.00 plus $4.00 or$14.00.

In a sixth scenario, contemplate that content provider B is awardedcontent item A1 (which it values at $4.00). Content items A2, B1, and B2remain to be allocated. As content provider E bid only on thecombination of all content items, the remaining content items to beallocated are of no interest to it. Further contemplate that contentprovider D is awarded the combination of content items A2 and B2 (whichit values at $6.00). Content item B1 remains to be allocated. As contentprovider C placed bids only on the combination of content items A1 andA2 or B1 and B2, the remaining content item is of no interest to it.Further, since it is assumed for purposes of this example that eachcontent provider may be awarded only a single content item orcombination of items on which it placed a bid, the only remaining optionis to award content item B1 to content provider A (which it values at$3.00). Thus, the overall value of this sixth scenario is $4.00 plus$6.00 plus $3.00 or $13.00.

In a seventh scenario (beginning the same as the sixth scenario),contemplate that content provider B is awarded content item A1 (which itvalues at $4.00). Content items A2, B1, and B2 remain to be allocated.As content provider E bid only on the combination of all content items,the remaining content items to be allocated are of no interest to it.Further contemplate that content provider C is awarded the combinationof content items B1 and B2 (which it values at $7.00). Content item A2remains to be allocated. As content provider D placed bids only on thecombination of content items A1 and B1 or A2 and B2, the remainingcontent item is of no interest to it. Further, since it is assumed forpurposes of this example that each content provider may be awarded onlya single content item or combination of items on which it placed a bid,the only remaining option is to award content item A2 to contentprovider A (which it values at $3.00). Thus, the overall value of thisseventh scenario is $4.00 plus $7.00 plus $3.00 or $14.00.

In an eighth scenario, contemplate that content provider B is awardedcontent item A2 (which it values at $2.00). Content items A1, B1, and B2remain to be allocated. As content provider E bid only on thecombination of all content items, the remaining content items to beallocated are of no interest to it. Further contemplate that contentprovider D is awarded the combination of content items A1 and B1 (whichit values at $6.00). Content item B2 remains to be allocated. As contentprovider C placed bids only on the combination of content items A1 andA2 or B1 and B2, the remaining content item is of no interest to it.Further, since it is assumed for purposes of this example that eachcontent provider may be awarded only a single content item orcombination of items on which it placed a bid, the only remaining optionis to award content item B2 to content provider A (which it values at$3.00). Thus, the overall value of this eighth scenario is $2.00 plus$6.00 plus $3.00 or $11.00.

In a ninth scenario (beginning the same as the eighth scenario),contemplate that content provider B is awarded content item A2 (which itvalues at $2.00). Content items A1, B1, and B2 remain to be allocated.As content provider E bid only on the combination of all content items,the remaining content items to be allocated are of no interest to it.Further contemplate that content provider C is awarded the combinationof content items B1 and B2 (which it values at $7.00). Content item A1remains to be allocated. As content provider D placed bids only on thecombination of content items A1 and B1 or A2 and B2, the remainingcontent item is of no interest to it. Further, since it is assumed forpurposes of this example that each content provider may be awarded onlya single content item or combination of items on which it placed a bid,the only remaining option is to award content item A1 to contentprovider A (which it values at $3.00). Thus, the overall value of thisninth scenario is $2.00 plus $7.00 plus $3.00 or $12.00.

In a tenth scenario, contemplate that content provider B is awardedcontent item B1 (which it values at $4.00). Content items A1, A2, and B2remain to be allocated. As content provider E bid only on thecombination of all content items, the remaining content items to beallocated are of no interest to it. Further contemplate that contentprovider D is awarded the combination of content items A2 and B2 (whichit values at $6.00). Content item A1 remains to be allocated. As contentprovider C placed bids only on the combination of content items A1 andA2 or B1 and B2, the remaining content item is of no interest to it.Further, since it is assumed for purposes of this example that eachcontent provider may be awarded only a single content item orcombination of items on which it placed a bid, the only remaining optionis to award content item A1 to content provider A (which it values at$3.00). Thus, the overall value of this tenth scenario is $4.00 plus$6.00 plus $3.00 or $13.00.

In an eleventh scenario (beginning the same as the tenth scenario),contemplate that content provider B is awarded content item B1 (which itvalues at $4.00). Content items A1, A2, and B2 remain to be allocated.As content provider E bid only on the combination of all content items,the remaining content items to be allocated are of no interest to it.Further contemplate that content provider C is awarded the combinationof content items A1 and A2 (which it values at $7.00). Content item B2remains to be allocated. As content provider D placed bids only on thecombination of content items A1 and B1 or A2 and B2, the remainingcontent item is of no interest to it. Further, since it is assumed forpurposes of this example that each content provider may be awarded onlya single content item or combination of items on which it placed a bid,the only remaining option is to award content item B2 to contentprovider A (which it values at $3.00). Thus, the overall value of thiseleventh scenario is $4.00 plus $7.00 plus $3.00 or $14.00.

In a twelfth scenario, contemplate that content provider A is awardedcontent item A1 (which it values at $3.00). Content items A2, B1, and B2remain to be allocated. As content provider E bid only on thecombination of all content items, the remaining content items to beallocated are of no interest to it. Further contemplate that contentprovider D is awarded the combination of content items A2 and B2 (whichit values at $6.00). Content item B1 remains to be allocated. As contentprovider C placed bids only on the combination of content items A1 andA2 or B1 and B2, the remaining content item is of no interest to it.Further, since it is assumed for purposes of this example that eachcontent provider may be awarded only a single content item orcombination of items on which it placed a bid, the only remaining optionis to award content item B1 to content provider B (which it values at$4.00). Thus, the overall value of this twelfth scenario is $3.00 plus$6.00 plus $4.00 or $13.00.

In a thirteenth scenario (beginning the same as the twelfth scenario),contemplate that content provider A is awarded content item A1 (which itvalues at $3.00). Content items A2, B1, and B2 remain to be allocated.As content provider E bid only on the combination of all content items,the remaining content items to be allocated are of no interest to it.Further contemplate that content provider C is awarded the combinationof content items B1 and B2 (which it values at $7.00). Content item A2remains to be allocated. As content provider D placed bids only on thecombination of content items A1 and B1 or A2 and B2, the remainingcontent item is of no interest to it. Further, since it is assumed forpurposes of this example that each content provider may be awarded onlya single content item or combination of items on which it placed a bid,the only remaining option is to award content item A2 to contentprovider B (which it values at $2.00). Thus, the overall value of thisthirteenth scenario is $3.00 plus $7.00 plus $2.00 or $12.00.

In a fourteenth scenario, contemplate that content provider A is awardedcontent item B1 (which it values at $3.00). Content items A1, A2, and B2remain to be allocated. As content provider E bid only on thecombination of all content items, the remaining content items to beallocated are of no interest to it. Further contemplate that contentprovider D is awarded the combination of content items A2 and B2 (whichit values at $6.00). Content item A1 remains to be allocated. As contentprovider C placed bids only on the combination of content items A1 andA2 or B1 and B2, the remaining content item is of no interest to it.Further, since it is assumed for purposes of this example that eachcontent provider may be awarded only a single content item orcombination of items on which it placed a bid, the only remaining optionis to award content item A1 to content provider B (which it values at$4.00). Thus, the overall value of this fourteenth scenario is $3.00plus $6.00 plus $4.00 or $13.00.

In a fifteenth scenario (beginning the same as the fourteenth scenario),contemplate that content provider A is awarded content item B2 (which itvalues at $3.00). Content items A1, A2, and B1 remain to be allocated.As content provider E bid only on the combination of all content items,the remaining content items to be allocated are of no interest to it.Further contemplate that content provider C is awarded the combinationof content items B1 and B2 (which it values at $7.00). Content item A2remains to be allocated. As content provider D placed bids only on thecombination of content items A1 and B1 or A2 and B2, the remainingcontent item is of no interest to it. Further, since it is assumed forpurposes of this example that each content provider may be awarded onlya single content item or combination of items on which it placed a bid,the only remaining option is to award content item A2 to contentprovider B (which it values at $2.00). Thus, the overall value of thisfifteenth scenario is $3.00 plus $7.00 plus $2.00 or $12.00.

As can be seen from the above-described scenarios, value is maximized at$14.00 if allocation is determined in accordance with any of scenarios4, 5, 7, or 11. As any of these scenarios will maximize value to thehosting site (e.g., the search engine web site), allocation may bedetermined among these scenarios according to any other criteria thehosting site so chooses, e.g., maximizing the number of contentproviders whose content items are shown or rewarding content providershaving a long history with the hosting site.

Selection and placement of content items in the content items positionson a rendered page (referred to herein as a “matrix”) may be moregenerally described with reference to FIG. 19. FIG. 19 is a flow diagramshowing a computer-implemented method for selecting content items to beplaced in a matrix on a rendered page, the content items having varyingshapes and/or sizes, in accordance with an embodiment of the presentinvention. Initially, as indicated at block 1910, the content items andtheir respective bid information may be organized such that all relevantinformation may be more easily evaluated. Such organization may take anynumber of forms known to those of ordinary skill in the art and themanner in which such organization takes place, or the form in which theorganized data may be rendered, are not intended to limit the scope ofthe present invention.

By way of example only, contemplate the seven content providers placebids for content items to be placed in a matrix on a rendered page, therendered page having a grid-like layout as shown in FIG. 10. As thematrix shown in FIG. 10 is two dimensional, the data may be organizationin terms of the number of slots that are necessary for the content itemsfor each of the x dimension and the y dimension. The exemplary contentitems and their respective bid values per slot are indicated in TableIII. TABLE III Content # of X # of y $ Per Item ID Axis Slots Axis SlotsTotal $ Slot 1 1 1 $3.00 $3.00 2 2 1 $6.00 $3.00 3 1 2 $7.00 $3.50 4 2 2$10.00 $2.50 5 1 2 $6.00 $3.00 6 1 1 $4.00 $4.00 7 1 1 $1.00 $1.00

Subsequently, as shown at block 1912, the content item having thehighest bid value per matrix slot is determined for placement in thefirst matrix slot to be filled. In one embodiment, the slots may befilled from left to right and from top to bottom. Thus, in the matrix ofFIG. 10, the first slot to which a content item will be awarded is slotA1. The content item chosen to fill slot A1 will be the slot providingthe maximum value therefore. Accordingly, as $4.00 per slot for contentitem ID 6 provides the highest value, it will be selected for slot A1and, accordingly, awarded position A1. This is indicated at block 1914.

Subsequently, as shown at block 1916, the remaining matrix area isdetermined. Next, the content items exceeding the remaining matrix areaand the content item awarded the first matrix slot (i.e., content itemID 6) are filtered out of consideration, as indicated at block 1918.

Subsequently, as the slots are being filled from left to right and fromtop to bottom, the second slot to be filled will be slot B1.Accordingly, it is next determined which of the remaining content itemsprovides the highest bid value for slot B1. This is indicated at block1920. Referring back to Table III, it can be seen that content item ID 3provides $3.50 value per slot. Since content item ID 3 requires one slotin the x dimension and two slots in the y dimension, such content itemwill fit within the remaining matrix area. Accordingly, as indicated atblock 1922, content item ID 3 is awarded the second matrix slot (slotB1). However, as content item ID 3 fills two slots in the y dimension,it also will be awarded the third matrix slot or slot B2.

It should be noted that if content item ID 3 had filled only slot B1, inthe present example, the next slot to be filled would have been slot B2.As such, the remaining matrix area would have been determined (asindicated at block 1924), the content items exceeding the remainingmatrix area and the content item awarded slot B2 would have beenfiltered out from consideration (as indicated at block 1926), theremaining content item providing the highest bid value for slot B2 wouldhave been determined (as indicated at block 1928) and, accordinglyawarded thereto (as indicted at block 1930). However, as content item ID3 filled both matrix slots B1 and B2, steps 1924, 1926, 1928, and 1930of the computer-implemented method of FIG. 3 may be effectively skippedin the present example.

Subsequently, the remaining matrix area is again determined, asindicated at block 1932, content item slot A2 being the only slot whichremains to be filled. The content items exceeding the remaining matrixarea (that is, those content items requiring more than one slot in the xdimension and one slot in the y dimension) are filtered out ofconsideration as are those content items which have already been awardedto a matrix slot (i.e., content item IDs 3 and 6). This is indicated atblock 1934. The remaining content item having the highest bid value forthe remaining matrix slot is subsequently determined, as indicated atblock 1936, and the fourth matrix slot is awarded thereto, as indicatedat block 1938.

In the present example, content item ID 1 provides a value of $3.00 forthe remaining slot and is, accordingly, awarded thereto. As such, thetotal value provided by the content items placed in the matrix inaccordance with the method of FIG. 19 and the matrix of FIG. 10 is$4.00+$7.00+$3.00 or $14.00.

Methods for Price and Position Estimation for Content Items to be Placedon a Rendered Page

As previously discussed, in the prior art methods for determiningrelative placement of content items (e.g., advertisements and/or images)on a rendered page (e.g., a web page), content providers are unable toview the bids that are being placed by their competitors. As such, theonly way that content providers can determine how their bids compare tothe bids being placed by their competitors is by examining the positionthey are ultimately awarded in the results listing relative to othercontent providers as a result of an appropriate query.

The present invention relates to systems and methods for estimating therelative placement position of a particular content item based upon aproposed bid and displaying such position estimation to a contentprovider. Further, the present invention relates to systems and methodsfor estimating the value a content provider would have to bid to have ahigh likelihood of having its content item placed in a desired contentitem position and displaying the bid pricing estimation to the contentprovider. The estimation methods and systems of the present inventionare extremely valuable to content providers as they permit contentprovider's to gain approximate knowledge of pricing and positioninformation before a page is rendered according to a given context.Without this knowledge, the content providers generally have no ideawhat the going rate is for a set of keywords and may only be able togather such information on their own after a long series of trial anderror.

With reference to FIG. 11, a flow diagram is illustrated which shows acomputer-implemented method 1100 for estimating the position on arendered page where a content item is likely to appear in accordancewith an embodiment of the present invention. Initially, as shown atblock 1110, the system receives a base bid for placement of a bidder'scontent item on a rendered page.

Referring to FIG. 12, an illustrative screen display 1200 of anexemplary user interface (UI) is shown on which such base bids (andtheir associated search phrases or bid terms and match type) may beentered or selected by a bidder and displayed. It will be understood andappreciated by those of ordinary skill in the art that this is only onepossible visual configuration and, accordingly, is not intended to limitthe scope of the invention in any way. A number of conceivableconfigurations may be presented which permit the bidder (i.e., thecontent provider) to enter the desired information and all suchvariations are contemplated to be within the scope of the presentinvention.

Screen display 1200 includes a search phrase entry/display area 1210, amatch type indication/display area 1212, and a base bid entry/displayarea 1214. The search phrase entry/display area 1210 permits a bidder(e.g., a content provider) to enter the search phrase or bid term(s) forwhich it desires to place a bid. In screen display 1200, the bidder hasentered the search phrase “Shoes”. The match type indication/displayarea 1212 permits a bidder to indicate in which type of match it desiresthe entered search term(s) or phrase to be included. A “broad” match isindicated in the match type indication/display area 1212 of screendisplay 1200. The base bid entry/display area 1214 permits the bidder toenter the base bid for which it wishes to receive a position estimationfor the combination of search term(s) or phrase entered and theindicated match type.

Referring back to FIG. 11, it is subsequently determined whether or notattribute targeting (as hereinabove described) is desired, as shown atblock 1112. If attribute targeting is not desired, the system determinesa bid value based solely on the base bid, as indicated at block 1114.

With reference to the screen display 1200 of FIG. 12, if attributetargeting is not desired, the bidder may select the submit indicator1216 (having an equal sign thereon) to indicate to the system that ithas entered all information it wishes to be considered in the positionestimation determination. Upon such user selection, the system willdetermine the bid value as indicated (FIG. 11).

Returning to FIG. 11, if attribute targeting is desired, the systemreceives an indication from the user of the attributes for whichtargeting is desired, as well as an indication of the bid adjustmentsassociated therewith. This is indicated at block 1116.

With reference again to the exemplary screen display 1200 of FIG. 12, ifthe bidder desires attribute targeting, it may select the advanced pricecalculator options area 1220 to view the attributes and attributeoptions based upon which it may target its bid. Once the advanced pricecalculator options area 1220 has been selected, the bidder may then bepresented with an additional exemplary screen display 1300, as shown inFIG. 13.

The exemplary screen display 1300 includes a search phrase entry/displayarea 1310, a match type indication/display area 1312, and a base bidentry/display area 1314, each of which permits the same functions andincludes the same information as the similarly delineated screen displayareas of the screen display 1200 of FIG. 12. Screen display 1300 furtherincludes an advanced price calculator options area 1318 that is expandedfrom that shown in the screen display 1200 of FIG. 12 such that itincludes a number of potential targeting attribute display areas. Forinstance, the screen display 1300 of FIG. 13 includes an age grouptargeting display area 1320, a gender targeting display area 1322, adaypart targeting display area 1324, a weekday targeting display area1326, a country targeting display area 1328, and a metro area targetingdisplay area 1330. Beneath each of the targeting display areas is anindication of the attribute options upon which the bidder has selectedtargeting to be based. For instance, in the screen display 1300 of FIG.13, the bidder has indicated in the age group targeting display area1320 that it wishes to target the age group 18-24, the age group 25-36and the age group 36-50.

It will be understood and appreciated by those of ordinary skill in theart that the potential targeting attribute display areas illustrated inscreen display 1300, as well as the attribute options shown associatedtherewith, are exemplary only and are not intended to limit the scope ofthe present invention in any way. It will be further understood andappreciated that the down arrow to the right of each of the attributetargeting display areas indicates that upon selection thereof, thebidder may be permitted to view a predefined list of attribute optionsfrom which it may choose. Alternatively, the user may enter theinformation directly in the attribute targeting display area, ifdesired.

Once the bidder has selected each of the attributes and attributeoptions on which it wishes to target its bid, it may select the submitindicator 1316 (having an equal sign thereon) to indicate to the systemthat it has entered all information it wishes to be considered in theposition estimation determination. The bidder will subsequently bereturned to the screen display shown in FIG. 12.

Returning to FIG. 11, once the system has received an indication of alldesired targeting attributes and attribute options, a bid value for thetargeted bid is determined, as indicated at block 1118. Subsequently,the system determines a position estimate based upon the bid valuedetermined (whether targeted or untargeted), as indicated at block 1120.

With reference to FIG. 14, a flow diagram is illustrated which shows acomputer-implemented method 1400 for determining a position estimatebased upon a determined bid value in accordance with an embodiment ofthe present invention. Initially, as shown at block 1410, the systemreceives a query from a content provider for which the content providerwould like a position estimation. For instance, contemplate that aparticular content provider submits a query for the bid terms “runningshoes” as a cost-per-click bid of $2.00 for a broad match type.

Subsequently, as indicated at block 1412, the system initializes theestimated click-through-rate (CTR) based on the same assumptions thatwould be used if the content provider were to place an order based onthe query received. The initial CTR may be estimated in a variety ofways, a number of which were discussed hereinabove.

Subsequently, a listing of keywords that are likely to compete with thereceived query is determined. This is indicated at block 1414. Toaccomplish this, the system initially may break the bid terms/phrasesinto one or more component terms/phrases (e.g., “Running”, “Shoes”,“Running Shoes”). Subsequently, the system may locate competingkeywords, e.g., by examining the query traffic for the componentterms/phrases over a past particular time frame (e.g., one month) andfinding all orders matching the component terms/phrases over the timeframe. Subsequently, as indicated at block 1416, the system may receiveinformation concerning impression data for the matching componentterms/phrases over the past time frame, as well as informationconcerning the bid values of, for instance, the top twenty positions formatching component terms/phrases over the past particular time frame.This is indicated at block 1418.

Subsequently, the system compares the proposed bid value to the actualbid value data, as indicated at block 1420, and determines a proposedestimated position. This is indicated at block 1422. This system thendisplays the proposed estimated position to the bidder as indicated atblock 1424 of FIG. 14, as well as block 1122 of FIG. 11.

In the illustrative screen display of FIG. 12, the estimated position atwhich the bidder's content item is likely to be displayed is indicatedin the estimated position display area 1218 of screen display 1200. Inthis example, it is estimated that a bid of $1.50 for the search phrase“shoes” in a broad match will permit the bidder's content item to bedisplayed at position 1.4.

With reference to FIG. 17, a flow diagram is illustrated which shows acomputer-implemented method 1100 for estimating the value a contentprovider would have to bid to have a high likelihood of having itscontent item placed in a desired content item position and displayingthe bid value/pricing estimation to the content provider. Initially, asshown at block 1710, the system receives an indication from the bidderconcerning in which position it would like its content item to appear,as well as an indication of the frequency at which it would like itscontent item to appear in the desired position.

Referring to FIG. 15, an illustrative screen display 1500 of anexemplary user interface (UI) is shown on which such desired positioninformation (and its associated search phrases or bid terms and matchtype) may be entered or selected by a bidder and displayed. It will beunderstood and appreciated by those of ordinary skill in the art thatthis is only one possible visual configuration and, accordingly, is notintended to limit the scope of the invention in any way. A number ofconceivable configurations may be presented which permit the bidder(i.e., the content provider) to enter the desired information and allsuch variations are contemplated to be within the scope of the presentinvention.

Screen display 1500 includes a search phrase entry/display area 1510, amatch type indication/display area 1512, a desired position indicationarea 1514, and a desired win percentage indication area 1516. The searchphrase entry/display area 1510 permits a bidder (e.g., a contentprovider) to enter the search phrase or bid term(s) for which it desiresto have its content item associated. In screen display 1500, the bidderhas entered the search phrase “Shoes”. The match type indication/displayarea 1512 permits the bidder to indicate in which type of match itdesires the entered search term or phrase to be included. A “broad”match is indicated in the desired match type indication/display area1512 of screen display 1500. The desired position indication area 1514permits the bidder to indicate the position in which it desires itscontent item to be placed on the rendered page and the desired winpercentage indication area 1516 permits the bidder to indicate theminimum frequency with which it would like its content item to appear inthe indicated position. In screen display 1500, the bidder has indicatedthat it desires its content item to be placed in position 1 at least 80%of the time.

Referring back to FIG. 17, it is subsequently determined whether or notattribute targeting (as hereinabove described) is desired, as shown atblock 1712. If attribute targeting is not desired, the system determinesan estimated bid value/price that the content provider must bid toattain its desired positioning and frequency, as more fully describedbelow. This is indicated at block 1716.

With reference to the screen display 1500 of FIG. 15, if attributetargeting is not desired, the bidder may select the submit indicator1518 (having an equal sign thereon) to indicate to the system that ithas entered all information it wishes to be considered in the priceestimation determination.

Returning to FIG. 17, if attribute targeting is desired, the systemreceives an indication from the user of the attributes for whichtargeting is desired. This is indicated at block 1714. With referenceagain to the exemplary screen display 1500 of FIG. 15, if the bidderdesires attribute targeting, it may select the advanced price calculatoroptions area 1522 to view the attributes and attribute options basedupon which it may target its bid. Once the advanced price calculatoroptions area 1522 has been selected, the bidder may then be presentedwith an additional exemplary screen display 1600, as shown in FIG. 16.

The exemplary screen display 1600 includes a search phrase entry/displayarea 1610, a match type indication/display area 1612, a desired positionindication area 1614, and a desired win percentage indication area 1616,each of which permits the same functions and includes the saeminformation as the similarly delineated screen display areas of thescreen display 1500 of FIG. 15. Screen display 1600 further includes anadvanced price calculator options area 1620 that is expanded from thatshown in the screen display 1500 of FIG. 15 such that it includes anumber of potential targeting attribute display areas. For instance, thescreen display 1600 of FIG. 16 includes an age group targeting displayarea 1622, a gender targeting display area 1624, a daypart targetingdisplay area 1626, a weekday targeting display area 1628, a countrytargeting display area 1630, and a metro area targeting display area1632. Beneath each of the targeting display areas is an indication ofthe attribute options upon which the bidder has selected targeting to bebased. For instance, in the screen display 1600 of FIG. 16, the bidderhas indicated in the weekday targeting display area 1628 that it wishesto target the weekdays Monday, Tuesday, and Friday.

It will be understood and appreciated by those of ordinary skill in theart that the potential targeting attribute display areas illustrated inscreen display 1600, as well as the attribute options shown associatedtherewith, are exemplary only and are not intended to limit the scope ofthe present invention in any way. It will be further understood andappreciated that the down arrow to the right of each of the attributetargeting display areas indicates that upon selection thereof, thebidder may be permitted to view a predefined list of attribute optionsfrom which it may choose. Alternatively, the user may enter theinformation directly in the attribute targeting display area, ifdesired.

Once the bidder has selected each of the attributes and attributeoptions on which it wishes its bid value/pricing estimate to be based,it may select the submit indicator 1618 (having an equal sign thereon)to indicate to the system that it has entered all information it wishesto be considered in the pricing estimation determination. The bidderwill subsequently be returned to the screen display shown in FIG. 15.Subsequently, returning to FIG. 17, the system determines an estimatedbid value/price that the content provider must bid to attain its desiredpositioning and frequency, as indicated at block 1716.

With reference to FIG. 18, a flow diagram is illustrated which shows acomputer-implemented method 1800 for determining a bid value/pricingestimate based upon a desired content item position and frequency inaccordance with an embodiment of the present invention. Initially, asshown at block 1810, the system receives a query from a content providerfor which the content provider would like a bid value/price estimation.For instance, contemplate that a particular content provider submits aquery for the bid terms “running shoes” for position 1, 80% of the time,for a broad match type.

Subsequently, as indicated at block 1812, the system determines alisting of keywords that are likely to compete with the received query.To accomplish this, the system initially may break the bid terms/phrasesinto one or more component terms/phrases (e.g., “Running”, “Shoes”,“Running Shoes”). Subsequently, the system may locate competingkeywords, e.g., by examining the query traffic for the componentterms/phrases over a past particular time frame (e.g., one month) andfinding all orders matching the component terms/phrases over the pasttime frame. Subsequently, as indicated at block 1814, the system mayreceive information concerning impression data for the matchingcomponent terms/phrases over the past time frame, as well as informationconcerning the bid values for, e.g., all matching keywords that appearedin the desired position over the past particular time frame (asindicated at block 1816) and information concerning the frequency withwhich all matching keywords appeared in the desired position over thepast particular time frame (as indicated at block 1818).

Subsequently, the system determines a proposed bid value/priceestimation, as indicated at block 1820, and displays the proposed bidvalue/price estimation to the bidder. This is indicated at block 1822 ofFIG. 18, as well as block 1718 of FIG. 17.

In the illustrative screen display 1500 of FIG. 15, the proposed bidvalue/price estimation is indicated in the estimated bid value/pricedisplay area 1520. In this example, it is estimated that a bid of $1.70for the search phrase “shoes” in a broad match will permit the bidder'scontent item to be displayed in position 1 about 80% of the time.

As can be understood, the present invention provides systems and methodsfor determining the placement of content items, e.g., advertisementsand/or images, on a rendered page, e.g., a web page, relative to othercontent displayed on the page. Further, the present invention relates tosystems and methods for determining relative placement of content itemsbased upon bid value. Bid values may be measured in terms of dollarspaid to a hosting web site, e.g., a search engine web site, there beingvarious different pricing models that may be mixed together in aprincipled manner to rank content descriptors according to value. Suchpricing models may differ among content items or with regard to a singlecontent item as its performance is evaluated over time. One such pricingmodel is a cost-per-click (CPC) pricing model under which a bid valuemay be determined by multiplying a CPC bid by the click-through-rate.The present invention further relates to systems and methods forestimating click-through-rate based, in part, upon observed data.

The present invention further provides systems and methods forintroducing randomness into the relative placement of content itemsbased upon bid value. Additionally, the present invention providessystems and methods for determining bid values based upon one or moretargeting attributes for which a bidder may desire to place anincrementally different bid. Further, the present invention providessystems and methods for determining the relative placement of contentitems of varying shapes and sizes based upon bid value. The presentinvention further provides systems and methods for estimating therelative placement position of a particular content item based upon aproposed bid and displaying such position estimate to a contentprovider. Additionally, the present invention provides systems andmethods for estimating the value a content provider would have to bid tohave a high likelihood of having its content item placed in a desiredcontent item position and displaying the bid value/pricing estimate tothe content provider.

The present invention has been described in relation to particularembodiments, which are intended in all respects to be illustrativerather than restrictive. Alternative embodiments will become apparent tothose of ordinary skill in the art to which the present inventionpertains without departing from its scope.

From the foregoing, it will be seen that this invention is one welladapted to attain all the ends and objects set forth above, togetherwith other advantages which are obvious and inherent to the system andmethod. It will be understood that certain features and subcombinationsare of utility and may be employed without reference to other featuresand subcombinations. This is contemplated and within the scope of theclaims.

1. A computer-implemented method for determining relative placement ofcontent items based upon bid value, the content items to be placed on apage rendered according to a given context, comprising: receiving aplurality of bids each associated with a content item; and determining arelative placement of one or more of the content items on the pageaccording to Vickrey-like auction principles.
 2. Thecomputer-implemented method of claim 1, wherein a plurality of contentitem placement positions are available on the page and wherein each ofthe plurality of bids received may be applied to each of the pluralityof content item placement positions.
 3. The computer-implemented methodof claim 2, further comprising determining a bid value for each of theplurality of bids.
 4. The computer-implemented method of claim 3,wherein determining a relative placement of one or more of the contentitems on the page according to Vickrey-like auction principles comprisesdetermining relative placement of one or more of the content items onthe page according to Vickrey-like auction principles using the bidvalue determined for each of the plurality of bids.
 5. Thecomputer-implemented method of claim 4, further comprising determining aprice to be paid for each of the plurality of content item placementpositions.
 6. The computer-implemented method of claim 5, whereindetermining a price to be paid for each of the plurality of content itemplacement positions comprises: determining an altered relative placementof one or more of the content items on the page as though a bid had notbeen received for the content item that was determined to be placed in aparticular content item placement position for which price is beingdetermined; and determining the price to be paid for the particularcontent item placement position for which price is being determined asthe bid value for the particular content item placement positionaccording to the altered relative placement.
 7. The computer-implementedmethod of claim 5, wherein determining a price to be paid for each ofthe plurality of content item placement positions comprises: determininga list having the identity of all bidders who placed bids for aparticular content item placement position for which price is beingdetermined, including a bidder whose content item was determined to beplaced in the particular content item placement position; determining analtered list having the identity of all bidders who placed bids for theparticular content item placement position except for the bidder whosecontent item was determined to be placed in the particular content itemplacement position; determining a sum of the bid values for all bidsreceived from bidders on the altered list; determining an identity of abidder whose content item would be determined to be placed in theparticular content item position based upon the altered list;determining a second altered list having the identity of all bidders whoplaced bids for the particular content item placement position exceptfor the bidder whose content item would be determined to be placed inthe particular content item placement position based upon the alteredlist; determining a sum of the bid values for all bids received frombidders on the second altered list; and determining a price to be paidfor each of the plurality of content item placement positions as the sumof the bid values for all bids received from bidders on the altered listless the sum of the bid values for all bids received from bidders on thesecond altered list.
 8. The computer-implemented method of claim 1,wherein a plurality of content item placement positions are available onthe page and wherein a plurality of different bids may be received froma single bidder, at least one of the plurality of different bids beingapplied to a different content item placement position than at least oneother of the plurality of different bids.
 9. The computer-implementedmethod of claim 1, further comprising displaying the one or more contentitems on the page according to the determined relative placement. 10.The computer-implemented method of claim 1, wherein receiving aplurality of bids comprises receiving at least one of the plurality ofbids through an auction.
 11. The computer-implemented method of claim 1,wherein the given context is determined based upon a search query.
 12. Acomputer programmed to perform the steps recited in claim
 1. 13. Acomputer-implemented method for determining relative placement ofcontent items based upon bid value, the content items to be placed on apage rendered according to a given context, comprising: receiving aplurality of bids each associated with a content item; and determining arelative placement of one or more of the content items on the pageaccording to modified Vickrey-like auction principles.
 14. Thecomputer-implemented method of claim 13, wherein a plurality of contentitem placement positions are available on the page and wherein each ofthe plurality of bids received may be applied to each of the pluralityof content item placement positions.
 15. The computer-implemented methodof claim 14, further comprising determining a bid value for each of theplurality of bids.
 16. The computer-implemented method of claim 15,further comprising determining an order of decreasing desirability amongthe plurality of content item placement positions.
 17. Thecomputer-implemented method of claim 16, wherein determining a relativeplacement of one or more of the content items on the page according tomodified Vickrey-like auction principles comprises determining therelative placement of the content items on the page according tomodified Vickrey-like auction principles using the bid value determinedfor each of the plurality of bids.
 18. The computer-implemented methodof claim 14, further comprising determining a price to be paid for eachof the plurality of content item placement positions.
 19. Thecomputer-implemented method of claim 13, further comprising displayingthe one or more content items on the page according to the determinedrelative placement.
 20. The computer-implemented method of claim 13,wherein receiving a plurality of bids comprises receiving at least oneof the plurality of bids through an auction.
 21. Thecomputer-implemented method of claim 13, wherein the given context isdetermined based upon a search query.
 22. A computer programmed toperform the steps recited in claim
 13. 23. A computer-readable mediumhaving computer-executable instructions embodied thereon for presentinga user interface on a display device, the user interface for determiningrelative placement of content items based upon bid value, the contentitems to be placed on a page rendered according to a given context,comprising: code for receiving a plurality of bids each associated witha content item; and code for determining a relative placement of one ormore of the content items on the page according to Vickrey-like auctionprinciples.
 24. The computer-readable medium of claim 23, furthercomprising code for determining a bid value for each of the plurality ofbids.
 25. The computer-readable medium of claim 24, wherein a pluralityof content item placement positions are available on the page, andwherein the computer-readable medium further comprises code fordetermining a price to be paid for each of the plurality of content itemplacement positions.
 26. The computer-readable medium of claim 23,further comprising code for displaying the one or more content items onthe page according to the determined relative placement.
 27. Acomputer-readable medium having computer-executable instructionsembodied thereon for presenting a user interface on a display device,the user interface for determining relative placement of content itemsbased upon bid value, the content items to be placed on a page renderedaccording to a given context, comprising: code for receiving a pluralityof bids each associated with a content item; and code for determining arelative placement of one or more of the content items on the pageaccording to modified Vickrey-like auction principles.
 28. Thecomputer-readable medium of claim 27, further comprising code fordetermining a bid value for each of the plurality of bids.
 29. Thecomputer-readable medium of claim 28, wherein a plurality of contentitem placement positions are available on the page, and wherein thecomputer-readable medium further comprises code for determining a priceto be paid for each of the plurality of content item placementpositions.
 30. A computer-implemented method for determining relativeplacement of content items of varying shapes and sizes based upon bidvalue, the content items to be placed on a page rendered according to agiven context, comprising: receiving a plurality of bids each associatedwith a content item; receiving an indication for which one or more of aplurality of content item placement positions each of the plurality ofbids is received; and determining a relative placement of one or more ofthe content items on the page.
 31. The computer-implemented method ofclaim 30, further comprising determining a bid value for each of theplurality of bids.
 32. The computer-implemented method of claim 31,wherein determining the relative placement of the one or more of thecontent items on the page comprises determining the relative placementof the one or more of the content items on the page such that an overallbid value is maximized.
 33. The computer-implemented method of claim 32,wherein determining the relative placement of the one or more of thecontent items on the page such that an overall bid value is maximizedcomprises determining the overall bid value as a sum of the bid valuefor each of the one or more the content items.
 34. Thecomputer-implemented method of claim 30, wherein receiving the pluralityof bids comprises receiving at least one of the plurality of bidsthrough an auction.
 35. The computer-implemented method of claim 30,further comprising displaying the one or more content items on the pagein accordance with the determined relative placement.
 36. Thecomputer-implemented method of claim 30, wherein the given context isdetermined based upon a search query.
 37. A computer programmed toperform the steps recited in claim
 30. 38. A computer-readable mediumhaving computer-executable instructions embodied thereon for presentinga user interface on a display device, the user interface for determiningrelative placement of content items of varying shapes and sizes basedupon bid value, the content items to be placed on a page renderedaccording to a given context, comprising: code for receiving a pluralityof bids each associated with a content item; code for receiving anindication for which one or more of a plurality of content itemplacement positions each of the plurality of bids is received; and codefor determining a relative placement of one or more of the content itemson the page.
 39. The computer-readable medium of claim 38, furthercomprising code for determining a bid value for each of the plurality ofbids.
 40. The computer-readable medium of claim 38, further comprisingcode for displaying the one or more of the content items on the page inaccordance with the determined relative placement.